* update openssl in cargo.toml, and remove dll and dylibs from repo
* cargo fmt
* cargo clippy
* cargo fmt again
* unused-mut
* adjust build.rs for test container
* windows runner setup
* fix masos linking
* same treatment for sbom files
* improve cftc endpoints and make a standalone workspace app
* mypy
* forgot couple of files to commit
* test params
* integration test param name
* fix sec-sic-search
* fill some contracts missing category and subcategory, add a workspace workchoices endpoint
* linter
* integration test for cot_choices
* fix(finra): use persistent session to prevent TooManyRedirects
Resolves#6972.
The FINRA OTC data fetching functions (get_finra_weeks, get_finra_data,
get_full_data) previously created new HTTP sessions for each request.
When making dozens of sequential API calls to aggregate weekly data,
this caused Cloudflare's bot protection on api.finra.org to trigger
redirect loops, resulting in a TooManyRedirects error.
Changes:
- Refactored get_finra_weeks and get_finra_data to accept an optional
session parameter via kwargs.
- Updated get_full_data to create a single requests.Session and perform
a pre-flight GET to finra.org to establish Cloudflare cookies before
making API calls.
- The session is reused across all requests within a single data fetch
operation, preventing the redirect loop.
- Removed hardcoded User-Agent from get_finra_data in favor of the
platform's standard user agent set by get_requests_session.
- Increased timeout for weeklyDownloadDetails from 3s to 20s for
consistency and reliability.
* Apply Black Formatting
* perf(finra): use async requests for concurrent weekly data fetching
Addresses reviewer feedback on PR #7443.
The get_full_data function previously made up to 52 sequential POST
requests to fetch weekly OTC data, resulting in poor performance.
This commit adds async equivalents (aget_finra_weeks, aget_finra_data,
aget_full_data) that use the platform's amake_request utility and
asyncio.gather to fetch all weeks concurrently through a shared
aiohttp session.
The FinraOTCAggregateFetcher now uses aextract_data, which is natively
supported by the platform's Fetcher base class.
The pre-flight GET to finra.org for Cloudflare cookie establishment
is preserved in the async path.
* fix(finra): return all rows from data fetching instead of just the first one
* run black, add symbol field to data model, and recapture test cassette
* mypy
---------
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
* add amendments and committees endpoints to congress_gov and migrate all items into the parent congress_gov folder
* codespell
* fix tests
* no-redef
* linting
* fix(sec): handle unexpected SEC API responses in cik_map and get_all_companies
Fixes#7097
The cik_map command throws 'ValueError: If using all scalar values,
you must pass an index' when the SEC API returns an unexpected response
format (e.g., flat dict instead of nested dict of dicts).
Changes:
- get_all_companies: validate response before DataFrame creation,
use from_dict(orient='index') instead of DataFrame().transpose(),
check column count matches expected fields
- cik_map fetcher: raise proper OpenBBError when symbol not found
instead of returning empty dict that passes truthiness checks
* run black
---------
Co-authored-by: JasonOA888 <JasonOA888@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
* fix: preserve comma-separated values for flagged CLI arguments
The CLI argument parser was splitting all comma-separated values into
separate positional args before argparse could process them. This caused
multi-symbol queries like --symbol AAPL,MSFT,GOOGL to fail with
'args couldn't be interpreted' for all symbols after the first.
Flag values are now identified by checking whether the preceding token
is a known option string with nargs != 0, and their commas are preserved
so the provider receives the original comma-separated string.
* test: add coverage for comma-split fix in parse_known_args_and_warn
---------
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
