Data Ingestion / ETL in Python

6 projects in Data & Science. Most starred: openbb, yfinance and akshare.

dlt is a Python library for loading data from APIs, files and databases into destinations such as DuckDB, BigQuery and Snowflake. It infers and evolves schemas, supports incremental loading and runs anywhere Python runs, with no server to operate. AWS SDK for pandas (awswrangler) connects DataFrames to Athena, Glue, Redshift and S3, which removes much of the glue code in AWS-based pipelines. To schedule and monitor these pipelines, see Job Schedulers for Airflow, Dagster and Prefect.

Several listed libraries focus on financial data. yfinance downloads market data from Yahoo Finance through unofficial endpoints, so it suits research rather than production systems. AKShare collects data for Chinese and other markets, edgartools downloads SEC EDGAR filings and parses XBRL financial statements, and OpenBB is a platform that brings many data providers behind one interface for analysts and agents.

General2

awswrangler

Pandas integration with AWS services like Athena, Glue, Redshift, S3, and DynamoDB.

4.1kupdated today3.17.1

dlt

A Python library for building data pipelines with automatic schema inference, incremental loading, and support for multiple sources and destinations.

6kupdated today1.31.0

Financial Data4

yfinance

Easy Pythonic way to download market and financial data from Yahoo Finance.

25.6kupdated today1.7.0

akshare

A financial data interface library, with data provided for academic research only.

23.1kupdated todayv1.19.3

edgartools

Downloads SEC EDGAR filings and XBRL financial statements as structured data.

2.8kupdated today5.62.1

openbb

A financial data platform for analysts, quants and AI agents.

74.6kupdated todayODP
About this site
Esc

Type to search every entry. ↑↓ to move, Enter to open.