Michigan Python Webinar - Data Analysis with DuckDB, 20240502
by pbeens@ · 25 things on Twos
- DuckDB is an in-memory analytical database that aims to provide fast query performance and low memory usage. It is designed to be used with Python and supports SQL queries.
- DuckDB is optimized for analytical workloads and can handle large datasets efficiently. It uses vectorized query execution and columnar storage to achieve high performance.
- StreamLit is a Python library that makes it easy to create interactive web applications for data science and machine learning. It allows you to quickly build and deploy web apps without needing to write HTML, CSS, or JavaScript.
- StreamLit provides a simple and intuitive API for creating interactive visualizations and dashboards. It supports various data visualization libraries such as Matplotlib, Plotly, and Altair.
- With StreamLit, you can easily turn your data analysis scripts into interactive web apps that can be shared and accessed by others. It provides features like caching, sharing, and deployment to make the process seamless.
- StreamLit also supports real-time data updates, allowing you to create dynamic and responsive web apps that can handle streaming data. This makes it suitable for applications like live dashboards and monitoring systems.
- Online Resources from Presentation
- LinkedIn
- Intro to DuckDB GitRepo
- Blog
- Slides
- GitHub Repo
- My Notes
- https://www.youtube.com/@MichiganPython
- Presented by Justin Smethers
- https://www.linkedin.com/in/justinsmethers/
- DuckDB Python API https://duckdb.org/docs/api/python/overview.html
- "SQLite for data analytics"
- Lightweight, no dependencies
- Fast and simple to use
- Billion line challenge: Pandas vs Polars vs DuckDB
- 1 Billion Row Challenge website: https://1brc.dev/
- awesome-duckdb on GitHub https://github.com/davidgasquez/awesome-duckdb
- A faster way to build and share data apps (Streamlit) https://streamlit.io/
- Streamlit