All 8 fixes have been applied to quanxiel/quantitative_data/importer.py and requirements.txt:
batch_insert (L113) — pd.NaT/NaN → None: Added df = df.where(pd.notna(df), None) before row extraction so psycopg2 receives Python None instead of unrecognizable NaT/NaN.
batch_insert (L120) — conflict_columns SQL identifier escaping: Replaced plain string concatenation conflict_str = ", ".join(conflict_columns) + sql.SQL(conflict_str) with sql.SQL(", ").join(map(sql.Identifier, conflict_columns)) to use proper sql.Identifier escaping for all conflict columns.
init_database (L915-930) — DDL errors: Changed from logger.debug + silent continue to logger.error + rollback + re-raise, so schema initialization failures are loudly visible. Each failing statement's SQL snippet (first 300 chars) is included in the error.
schema.sql splitting: Replaced naive ddl_sql.split(";") with sqlparse.split() when the library is available, with a graceful fallback to simple split. Added sqlparse>=0.4.0 to requirements.txt.
get_sqlalchemy_engine (L57-60): Already uses PASSWORD_ENCODED from config.py which applies quote_plus() to the raw password — no change needed.
import_daily_basic_by_date: Wrapped the per-date pro.daily_basic() call in fetch_with_retry(fetch_daily_basic, max_retries=3) for resilience against transient API failures.
import_financial_statements: Upgraded the per-stock per-table exception handler from logger.debug to logger.warning.
Type annotations: Fixed all str = None → Optional[str] = None and List[str] = None → Optional[List[str]] = None across 11 function signatures (import_trade_cal, import_daily_batch, import_daily_basic, import_daily_basic_by_date, import_adj_factor, import_adj_factor_batch, import_financial_statements, import_index_daily, full_import).
All 8 fixes have been applied to `quanxiel/quantitative_data/importer.py` and `requirements.txt`:
1. **batch_insert (L113) — pd.NaT/NaN → None**: Added `df = df.where(pd.notna(df), None)` before row extraction so psycopg2 receives Python `None` instead of unrecognizable `NaT`/`NaN`.
2. **batch_insert (L120) — conflict_columns SQL identifier escaping**: Replaced plain string concatenation `conflict_str = ", ".join(conflict_columns)` + `sql.SQL(conflict_str)` with `sql.SQL(", ").join(map(sql.Identifier, conflict_columns))` to use proper `sql.Identifier` escaping for all conflict columns.
3. **init_database (L915-930) — DDL errors**: Changed from `logger.debug` + silent continue to `logger.error` + rollback + re-raise, so schema initialization failures are loudly visible. Each failing statement's SQL snippet (first 300 chars) is included in the error.
4. **schema.sql splitting**: Replaced naive `ddl_sql.split(";")` with `sqlparse.split()` when the library is available, with a graceful fallback to simple split. Added `sqlparse>=0.4.0` to `requirements.txt`.
5. **get_sqlalchemy_engine (L57-60)**: Already uses `PASSWORD_ENCODED` from `config.py` which applies `quote_plus()` to the raw password — no change needed.
6. **import_daily_basic_by_date**: Wrapped the per-date `pro.daily_basic()` call in `fetch_with_retry(fetch_daily_basic, max_retries=3)` for resilience against transient API failures.
7. **import_financial_statements**: Upgraded the per-stock per-table exception handler from `logger.debug` to `logger.warning`.
8. **Type annotations**: Fixed all `str = None` → `Optional[str] = None` and `List[str] = None` → `Optional[List[str]] = None` across 11 function signatures (`import_trade_cal`, `import_daily_batch`, `import_daily_basic`, `import_daily_basic_by_date`, `import_adj_factor`, `import_adj_factor_batch`, `import_financial_statements`, `import_index_daily`, `full_import`).
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
关联主报告：#1 —
quantitative_data/importer.py待办项
batch_insert约 L109-110 —pd.NaT/NaN直接入库psycopg2无法识别NaT，NaN写入数值列产生 PostgreSQL NaNdf.where(pd.notna(df), None)统一替换batch_insert约 L113-137 —conflict_columns未做 SQL 标识符转义sql.Identifier构造init_database()约 L915-930 — DDL 错误被静默吞掉（只记 debug）logger.error+ 重新抛出，避免 schema 不完整时难排查schema.sql按分号切分过脆 — 建议sqlparse.split或直接用psql执行get_sqlalchemy_engine()约 L57-60 — 密码未 URL 编码，用quote_plusimport_daily_basic_by_date()无重试机制，复用fetch_with_retry()import_financial_statements()异常只记 debug，改为 warning/error类型注解错误
str = None多处，改Optional[str] = NoneAll 8 fixes have been applied to
quanxiel/quantitative_data/importer.pyandrequirements.txt:batch_insert (L113) — pd.NaT/NaN → None: Added
df = df.where(pd.notna(df), None)before row extraction so psycopg2 receives PythonNoneinstead of unrecognizableNaT/NaN.batch_insert (L120) — conflict_columns SQL identifier escaping: Replaced plain string concatenation
conflict_str = ", ".join(conflict_columns)+sql.SQL(conflict_str)withsql.SQL(", ").join(map(sql.Identifier, conflict_columns))to use propersql.Identifierescaping for all conflict columns.init_database (L915-930) — DDL errors: Changed from
logger.debug+ silent continue tologger.error+ rollback + re-raise, so schema initialization failures are loudly visible. Each failing statement's SQL snippet (first 300 chars) is included in the error.schema.sql splitting: Replaced naive
ddl_sql.split(";")withsqlparse.split()when the library is available, with a graceful fallback to simple split. Addedsqlparse>=0.4.0torequirements.txt.get_sqlalchemy_engine (L57-60): Already uses
PASSWORD_ENCODEDfromconfig.pywhich appliesquote_plus()to the raw password — no change needed.import_daily_basic_by_date: Wrapped the per-date
pro.daily_basic()call infetch_with_retry(fetch_daily_basic, max_retries=3)for resilience against transient API failures.import_financial_statements: Upgraded the per-stock per-table exception handler from
logger.debugtologger.warning.Type annotations: Fixed all
str = None→Optional[str] = NoneandList[str] = None→Optional[List[str]] = Noneacross 11 function signatures (import_trade_cal,import_daily_batch,import_daily_basic,import_daily_basic_by_date,import_adj_factor,import_adj_factor_batch,import_financial_statements,import_index_daily,full_import).