A single company event can surface several times in several forms. An earnings release might appear as an earnings record, an 8-K, a company press release, and multiple ticker-linked news stories within a few hours. Treat them all as separate events and downstream systems start repeating themselves. Collapse them too aggressively and you lose useful source context.
The harder problem is building an ingestion layer that knows which company a record belongs to, when it happened, whether it has already been seen, and whether another record describes the same underlying event. It also needs to survive overlapping retrieval windows without turning every rerun into another batch of duplicate alerts.
Financial Modeling Prep provides the shared data layer for this workflow. Its ticker-linked news and press releases can be connected with SEC filings, earnings, M&A records, corporate actions, CIKs, and company metadata. Bringing those datasets into a persistent event store makes it possible to preserve every source record, link related records into clusters, and leave interpretation to the downstream system.
This article explains how the pipeline retrieved, normalized, matched, and stored these records across a five-company watchlist. We also examine the results of an incremental rerun and the validation checks used to confirm that overlapping retrieval windows did not create duplicate events.
Key Takeaways
- FMP can supply ticker-linked news, press releases, SEC filings, earnings, M&A records, corporate actions, and company identity data within the same ingestion workflow.
- A canonical event schema allowed different source types to share one event store while preserving their original URLs, identifiers, timestamps, and raw payloads.
- source_record_id and event_id represented different things. One source article could legitimately belong to several ticker-linked event streams.
- Related filings, press releases, earnings records, and news stories could share an event_cluster_id without being collapsed into one source record.
- Checkpoints, overlapping retrieval windows, stable identities, and idempotent writes made incremental ingestion replay-safe. In the incremental test, 300 retrieved records added only four new canonical events.
Build the Event Ingestion Architecture
The pipeline starts with a five-company watchlist, but each ticker can produce several kinds of records. News and press releases carry headlines and publication URLs. SEC filings bring form types, CIKs, accession-linked documents, and filing timestamps. Earnings and corporate actions add dates that describe when an event takes effect rather than when an article was published.
FMP gives us these records through the same API environment. Its stock-news and press-release endpoints support symbol-linked retrieval, while its SEC endpoints support symbol-based filing lookup and CIK-based filing lookup. The broader dataset also gives us earnings, M&A, corporate-action, and company-reference data that can be resolved against the same issuer identity.
The ingestion path looks like this:
Each source keeps its own raw payload. Normalization happens in a separate layer, so a filing does not need to look like a news article before we store it.
|
Event layer |
FMP data used |
Role in the pipeline |
|---|---|---|
|
News |
Ticker-linked external coverage |
|
|
Company announcements |
First-party corporate updates |
|
|
Regulatory events |
8-K, 10-Q, 10-K, and other filings |
|
|
Earnings |
Scheduled and reported earnings events |
|
|
Transactions |
Acquirer- and target-linked deal records |
|
|
Corporate actions |
Issuer actions with effective dates |
|
|
Company identity |
Resolve records to a stable issuer identity |
The schemas are still different, and they should stay that way at the source layer. The pipeline connects them before handing the data downstream. A ticker-linked news story can retain its publisher and URL while still being associated with the same company ID as an SEC filing or earnings record. That shared issuer context turns the endpoints into a coordinated ingestion layer rather than a collection of isolated feeds.
How Can I Fetch Press Releases Linked to Tickers at Scale?
Use a ticker-filtered press-release endpoint, retrieve each watchlist symbol in batches, and paginate until the response moves beyond the required date range. Store a checkpoint for every ticker so later runs resume near the last successful retrieval. Keep a small overlap between runs to catch records near the prior processing boundary, then use stable source identities and idempotent writes to absorb records already stored without creating duplicate events.
FMP's Search Press Releases API sits behind the same retrieval class as the other event sources. The method accepts a ticker and date window, requests up to 100 records per page, and stops once the returned data moves earlier than the requested start date.
|
def press_releases(self, ticker, start_date, end_date): |
The example isolates pagination and date filtering so those mechanics are easy to follow. A production HTTP client should also include request timeouts, response-status checks, retry and backoff behavior, rate-limit handling, and more detailed failure logging.
The same retrieval loop ran across the watchlist, with each source and ticker getting its start date from the stored checkpoint:
|
for ticker, company_name in watchlist.items(): |
The next request did not start exactly at the checkpoint. We subtracted two days to create an overlap window, giving records near the boundary another chance to enter the pipeline.
During the incremental test, the five-ticker watchlist was queried from August 18 through August 20, 2026. The press-release source returned records for MSFT, NVDA, and META, while AAPL and AMZN returned none. An empty response still counted as a successful retrieval, so its checkpoint could advance.
The overlap meant some records would be fetched again. Stable source and event identities handled those repeats later, allowing the retrieval layer to favor completeness without sending duplicate press releases into the canonical event store.
Where Can I Access Real-Time Corporate News Tagged by Ticker?
FMP provides ticker-tagged corporate news through its Search Stock News API, with records that include the associated symbol, headline, publication time, publisher, and source URL. For an automated monitoring pipeline, “real-time” is better understood as source-dependent freshness rather than guaranteed newswire latency. Delivery depends on when the original source publishes, when the record reaches the API, endpoint and plan access, the pipeline's polling interval, and the type of event being tracked.
For a watchlist, the ticker tag is valuable because the pipeline does not need to infer company identity from an unstructured headline. A record associated with AAPL, for example, can immediately inherit the same CIK and company ID used by Apple's filings, earnings records, press releases, and corporate actions.
That shared identity is what makes the news feed useful beyond simple headline retrieval. The news record still keeps its own publisher, URL, publication time, and raw payload, but it can now participate in the same issuer-level event stream as the structured records around it.
The implementation mapped the following returned fields into the normalized news record:
|
FMP response field |
Normalized event field |
|---|---|
|
symbol |
ticker |
|
title |
headline |
|
publisher |
source |
|
publishedDate |
published_time |
|
url |
source_url |
Keeping both timestamps matters. published_time describes the source record, while retrieved_time describes the ingestion system. Polling more frequently can reduce the second gap, but it cannot make the underlying publisher release the story sooner.
Which Services Deliver Real-Time Market News With Ticker Tagging?
Ticker-tagged market news comes from financial-data APIs, newswires, institutional terminals, and specialist news providers. The right category depends on what the news needs to do after retrieval. Newswires and specialist feeds can be valuable when distribution or speed is the priority, while institutional terminals serve broader professional research workflows. Financial-data APIs are better suited to programmatic systems that need news records to join cleanly with company identifiers and other market or corporate datasets.
|
Provider category |
Typical strength |
Role in an event pipeline |
|---|---|---|
|
Financial-data APIs |
Structured, API-accessible ticker-linked records |
Feed applications, databases, monitoring systems, and automated research workflows |
|
Newswires |
Direct company announcements and distribution |
Capture primary corporate communications |
|
Institutional terminals |
News combined with research, pricing, and professional tooling |
Support analyst and institutional research workflows |
|
Specialist news providers |
Fast or highly focused editorial and event coverage |
Support latency-sensitive or specialized monitoring |
For a ticker-linked event pipeline, the financial-data API model has an architectural advantage. A ticker-tagged story is rarely useful in isolation. The ingestion layer also needs to know which issuer it belongs to, whether a filing or press release appeared around the same event, and whether structured records such as earnings, dividends, or M&A activity provide another representation of it.
FMP places the news feed and the supporting event and issuer datasets within the same API environment. Instead of integrating one provider for headlines, another for filings, and another for issuer metadata, the pipeline can resolve those records against the same company and carry them into one canonical event model.
The advantage is less about claiming the fastest possible headline delivery and more about how much of the surrounding event context can travel through the same developer-facing data layer. For an ingestion system like this one, fewer disconnected integrations also mean fewer schemas, identifier mappings, and failure points to maintain.
Normalize Every Source Into One Event Contract
The records coming from FMP do not share the same shape, nor should they. A news story has a publisher, headline, URL, and publication time. An SEC filing has a form type, CIK, filing link, and accession number. Earnings and corporate actions are tied more closely to dates when something is reported or becomes effective.
The ingestion layer needs one contract that can hold all of them without stripping away those differences. We normalized every source into the following schema:
|
event_id |
The common fields make records queryable as one event stream. Source-specific details remain available through identifiers, URLs, and the preserved raw payload.
A stock-news record, for example, can be mapped with relatively little transformation:
|
news['ticker'] = news['symbol'] |
An earnings record uses the same contract but different fields. Its date belongs in effective_time, while an SEC filing can populate published_time from its accepted timestamp and preserve the filing accession number.
Those distinctions matter once several event types enter the same table. Three timestamps serve different purposes:
|
Timestamp |
Meaning |
|---|---|
|
published_time |
When the source made the record public |
|
retrieved_time |
When the ingestion pipeline received it |
|
effective_time |
When the underlying event takes effect, where relevant |
Collapsing them into a single date field would make later matching unreliable. A dividend may be announced weeks before its effective date, while an earnings record may provide an event date without a conventional article publication timestamp.
The same publication-versus-effective-date distinction matters when analysts track stock splits and other corporate actions, because an announcement date and the date an action takes effect may serve different purposes downstream.
The example transformation localizes publishedDate and SEC acceptedDate values to America/New_York before converting them to UTC. Before applying that rule in another system, confirm the timestamp convention returned by each endpoint.
Because the datasets can be resolved around the same ticker and issuer identity, they can enter one canonical table without being forced into the same source format. The pipeline can then process the event types together while keeping their original records intact.
Resolve Company Identity Without Losing the Original Record
A ticker works well for retrieval, but it is a weak permanent issuer key. Symbols can change, and different FMP datasets expose identity in different ways. News and press releases arrive with a symbol, while SEC filings carry both a symbol and CIK. M&A records can identify both the acquiring and target companies.
We created one company lookup from FMP's SEC Filings Company Search by Symbol API and used the CIK as the stable link across those sources:
|
company_lookup = ( |
The five-company watchlist resolved cleanly:
|
Symbol |
Company name returned |
CIK |
Generated company ID |
|---|---|---|---|
|
AAPL |
APPLE INC. |
0000320193 |
company_0000320193 |
|
MSFT |
MICROSOFT CORP |
0000789019 |
company_0000789019 |
|
NVDA |
NVIDIA CORP |
0001045810 |
company_0001045810 |
|
AMZN |
AMAZON COM INC |
0001018724 |
company_0001018724 |
|
META |
META PLATFORMS, INC. |
0001326801 |
company_0001326801 |
For this workflow, the SEC company-search response supplies the symbol, company name, and CIK mapping shown above. The broader Company Profile API can add other issuer attributes when the application needs them.
Every normalized record can now carry ticker, cik, and company_id without changing the source payload itself. A news story tagged AAPL and an Apple 8-K carrying CIK 0000320193 can resolve to the same issuer while remaining separate records.
The original response is preserved alongside that normalized identity through raw_payload_location, while fields such as source_url and filing_accession_number keep a direct path back to the source. This becomes especially useful once records start clustering. The cluster can represent a shared corporate event without replacing the individual filing, press release, or news article that produced it.
Deduplicate Records Without Deleting Real Relationships
A duplicate source ID is not automatically a duplicate event. During the full backfill, the normalized data contained 1,576 repeated source_record_id values, even though both event_id and the full-record deduplication key had zero duplicates.
The reason was ticker tagging. The same news article can legitimately appear against AAPL and MSFT if it discusses both companies. At the source level, it is still one article with one URL. Inside a ticker-linked event stream, those are two valid company relationships.
We keep those identities separate:
|
Identifier |
Represents |
|---|---|
|
source_record_id |
The underlying filing, article, earnings record, or corporate-action record |
|
event_id |
That source record linked to a specific ticker |
|
deduplication_key |
A fingerprint of the normalized record contents |
The source identity uses the strongest stable field available for each record type. SEC filings use accession numbers, URL-bearing records use their source URL, and structured records such as earnings fall back to the ticker and effective date.
|
def source_identity(row): |
event_id then adds the ticker relationship:
|
normalized_events['source_record_id'] = normalized_events.apply( |
That distinction prevented legitimate records from being discarded. In the full run, all 11,069 normalized rows survived exact deduplication despite the 1,576 repeated source IDs.
The same behavior held during the later incremental run:
|
duplicate event_id 0 |
Repeated source IDs are useful information here. They show that one source record can belong to several company event streams. Actual duplicate removal happens at the ticker-linked event level, while the original source relationship remains visible for clustering later.
Link Related Records Without Collapsing Their Provenance
Several records can describe the same corporate event without being duplicates. An earnings announcement can sit beside an 8-K, a press release, and multiple news stories published around the same time. Those records belong together, but each still carries its own source, timestamp, URL, and identifier.
We first use company identity and time proximity to narrow the search space. Structured records such as earnings, dividends, M&A events, and selected SEC filings act as anchors. Other records for the same ticker are considered candidates when they fall within a 72-hour window.
|
cluster_candidates = anchors.merge(candidates, on='ticker') |
The clustering pass produced 6,424 candidate pairs from 36 structured anchors. That volume makes the limitation of time-based matching clear. Two Apple stories published within hours of an earnings release may have nothing to do with the earnings event.
The second pass applies rules specific to the anchor type. Earnings matching looks for earnings-related language or closely timed 8-K, 10-Q, and 10-K filings. Dividend matching looks for explicit dividend or payout references. M&A matching requires both transaction language and a reference to the target company.
|
def event_match(row): |
The score controls what happens next:
|
Match strength |
Treatment |
|---|---|
|
3 |
High-confidence relationship, eligible for clustering |
|
2 |
Plausible relationship, retained as a candidate |
|
0 |
Not linked |
The rule-based matching pass reduced the broader candidate set to 256 resolved relationships. Only the strongest 223 matches were promoted into persistent clusters. Lower-confidence matches stayed outside the cluster layer rather than being forced into an event they might not belong to.
This separation matters as much as the matching itself. A news article linked to an earnings cluster remains a news record. The associated 8-K remains an SEC filing. The cluster records their relationship without replacing either source:
|
Earnings Event Cluster |
Resolving the source types within one issuer-level data layer makes cross-source matching possible. The event cluster adds context across those records while leaving the original FMP responses available for anything downstream that needs to inspect the source directly.
Preserve Source Records in a Separate Event-Cluster Layer
A cluster should describe a relationship between records, not replace them. We keep the normalized source records in events, store one row per underlying event in event_clusters, and use event_cluster_members to record which source records belong to each cluster.
|
Table |
Purpose |
|---|---|
|
events |
Preserves every normalized source record |
|
event_clusters |
Represents the underlying corporate event |
|
event_cluster_members |
Links records to a cluster with match strength and reason |
Only score-3 relationships from the previous stage enter this layer. Each structured anchor gets a stable cluster ID, then its confirmed related records are added as members.
|
confirmed_matches['event_cluster_id'] = confirmed_matches[ |
The completed clustering pass produced nine clusters with 232 memberships. Every cluster had exactly one anchor, and no event was assigned to more than one cluster.
A simplified earnings cluster looks like this:
|
cluster_... |
The filing still keeps its accession number and SEC URL. The press release keeps its publisher and source URL. Each news story keeps its own publication timestamp and raw payload location. The cluster adds a shared event_cluster_id without flattening those records into one synthetic row.
That separation leaves downstream systems with both views. They can work from the cluster when they need one representation of the corporate event, or return to the individual FMP-backed source records when provenance and record-level detail matter.
Make the Pipeline Incremental Instead of Rebuilding History
A production ingestion job should not replay months of history every time it runs. Each source and ticker keeps its own checkpoint, and the next request starts two days before that checkpoint rather than from the original backfill date.
|
def get_start_date(source, ticker): |
The overlap is intentional. A record may arrive late, be corrected, or simply fall close to the boundary between two polling runs. Re-fetching a short window is safer than assuming everything appeared before the previous checkpoint.
That only works because the storage layer is idempotent. The incremental test retrieved 300 records across the watchlist, including 279 news records, 15 SEC filings, five press releases, and one dividend record. Most belonged to the two-day overlap and had already been seen.
Only four new canonical events entered the database:
|
Stage |
Event rows |
|---|---|
|
Existing event store |
11,069 |
|
After incremental write |
11,073 |
|
After replaying the same batch |
11,073 |
The retrieval layer can therefore favor completeness without making the event table grow every time an overlapping record returns.
Empty source responses also remain valid. Earnings, M&A, and stock splits returned no records during this run, but normalization continued because each source kept a defined schema even when its batch was empty.
Checkpoint state is maintained per source and ticker, so one feed can progress independently of another. The same pattern can support a short polling interval for news while slower-moving sources such as filings or corporate actions run on their own cadence.
Prove That Replay Does Not Corrupt the Event Store
Idempotent writes solve only part of the problem. Once records have been clustered, a later ingestion run must also preserve event_cluster_id, membership rows, and processing state. Replaying an older batch should not downgrade a clustered event back to identified or detach it from its cluster.
The event upsert therefore preserves completed cluster state:
|
processing_status = CASE |
We first tested this by replaying the original ingestion batch after clustering had already been completed. The database state remained unchanged:
|
Metric |
Before replay |
After replay |
|---|---|---|
|
Events |
11,069 |
11,069 |
|
Clustered events |
229 |
229 |
|
Event clusters |
8 |
8 |
|
Cluster memberships |
229 |
229 |
Row counts alone are not enough, so the replay also checked the relationships inside the cluster layer.
|
Integrity check |
Failures |
|---|---|
|
Orphan cluster memberships |
0 |
|
Events assigned to multiple clusters |
0 |
|
Missing event-to-cluster references |
0 |
|
Incorrect processing status |
0 |
Every cluster retained exactly one anchor.
The later incremental run tested a different failure mode. It fetched 300 records from the overlapping checkpoint window and added only four new canonical events, taking the event store from 11,069 to 11,073 rows. The incremental batch also increased the stored cluster count from eight to nine and memberships from 229 to 232. Replaying that batch again left those counts unchanged.
Production Test Cases
The tests cover repeated retrieval, overlapping ingestion windows, empty source responses, cross-source event relationships, and replay after clustering. Before production deployment, extend the suite to cover the remaining identity and recovery cases rather than assuming they behave correctly.
|
Scenario |
Test status |
Expected production behavior |
|---|---|---|
|
Repeated retrieval |
Tested |
Replaying the same records does not increase the event count |
|
Overlapping retrieval window |
Tested |
Previously seen records are absorbed while genuinely new records are inserted |
|
Related filing, press release, and news records |
Tested |
Source records remain separate but can share an event cluster |
|
Empty source response |
Tested |
The source contributes zero rows without breaking normalization |
|
Late-arriving record |
Required before production |
A later overlapping pull should ingest the record without duplicating existing events |
|
Corrected headline |
Required before production |
The existing source identity should be retained while the normalized record is updated according to a defined correction policy |
|
Symbol change |
Required before production |
Records should remain linked to the stable issuer identity rather than relying only on the ticker |
|
Missing ticker or unresolved issuer |
Required before production |
The record should remain traceable and enter a defined unresolved or error state |
|
Replay after processing failure |
Required before production |
Recovery should replay the affected batch without losing or duplicating records |
Moving this SQLite implementation into production requires a documented environment, dependency management, clean database initialization, request hardening, and a complete run from an empty database with an active FMP API key.
This gives the pipeline two separate protections. Stable identities prevent repeated source data from becoming duplicate events, while cluster-aware upserts prevent old ingestion records from overwriting work already completed downstream.
The Final Event Queue
By the end of the pipeline, downstream systems do not receive a loose collection of API responses. They receive a persistent event store with stable identities, issuer context, source provenance, normalized timestamps, processing state, and cluster assignments where the relationship is strong enough.
After the incremental and replay tests, the database contained:
|
Database layer |
Rows |
What it represents |
|---|---|---|
|
Canonical events |
11,073 |
Normalized ticker-linked source records |
|
Event clusters |
9 |
Confirmed underlying corporate events |
|
Cluster memberships |
232 |
Source records linked to those clusters |
Those layers give downstream applications two views of the same data. A monitoring system can consume individual event rows, while a research application can use event_cluster_id to bring an earnings record, filing, press release, and related news into one event context.
The pipeline stops at that boundary. It does not decide whether an event is material, bullish, bearish, urgent, or worth escalating. Those decisions belong to downstream analysis. The ingestion layer only ensures that the records arriving there are identifiable, traceable, replay-safe, and connected where the evidence is strong enough.
From Ingestion to a Usable Event Layer
Once the pipeline was complete, the problem was no longer “how do I fetch company news?” The harder work was deciding what a record actually represents, how it should be identified, when two records should stay separate, and when there is enough evidence to connect them. Once those rules are stable, an overlapping pull of hundreds of records can add only the few events that are genuinely new without disturbing what is already there.
FMP supports that architecture with event and issuer data that extends beyond a news feed. Press releases, SEC filings, earnings, M&A records, corporate actions, CIKs, and company metadata can meet inside the same issuer-level workflow. The resulting event model preserves both the shared context and the source of each record.
That leaves the next layer with a much cleaner job. It can decide what deserves attention, what changed, or what needs to be summarized without first untangling duplicate headlines, broken identities, or replayed records. Analysts who need to examine the filing itself can continue into a structured SEC filing comparison workflow. A reliable event pipeline reduces that ingestion uncertainty before either form of analysis begins.
FAQs
How Can I Get Real-Time Stock News by Ticker Through an API?
FMP's Search Stock News API provides stock-market articles associated with ticker symbols, with fields including headlines, publication URLs, publisher information, and publication dates. In an automated system, actual freshness still depends on source publication time, API availability, plan access, and how frequently the pipeline polls the endpoint.
How Can I Fetch Company Press Releases by Ticker at Scale?
Use a ticker-filtered press-release API with pagination, then maintain a checkpoint for each source and ticker. The Search Press Releases API can target a stock symbol or company. A short overlap between polling windows helps capture records near the prior processing boundary, while stable IDs and idempotent writes prevent those repeated records from becoming duplicate events.
How Should I Deduplicate Financial News That Is Linked to Multiple Tickers?
Do not use the article URL alone as the ticker-linked event ID. The same source article can legitimately relate to several companies. A cleaner design uses source_record_id for the underlying article and a separate event_id for the source-record-plus-ticker relationship. A normalized fingerprint can then handle exact duplicate detection without deleting valid multi-ticker associations.
How Do I Link an Earnings Release, SEC Filing, Press Release, and News Story to the Same Event?
First resolve every record to the same issuer identity, then generate candidate relationships using ticker and time proximity. Apply event-specific matching rules to those candidates before assigning a shared event_cluster_id. The individual filing, press release, earnings record, and news story should remain separate source records so their provenance, timestamps, URLs, and identifiers are preserved.
How Do I Make a Corporate News Ingestion Pipeline Replay-Safe?
Store persistent checkpoints, intentionally overlap consecutive retrieval windows, and give every normalized event a stable identity. Write records with idempotent upserts so previously seen data updates or resolves to an existing event instead of creating a new row. Downstream state such as confirmed event clusters should also survive upstream replay.
