Skip to content

BrainCompanyFilingLanguageMetrics universe selection does not expose Similarity values #32

Description

@Ruuudy1

Expected Behavior

BrainCompanyFilingLanguageMetricsUniverseAll and BrainCompanyFilingLanguageMetricsUniverse10K should expose the filing language metric similarity values during universe selection, for example d.ReportSentiment.Similarity.All (and the risk factor and management discussion similarity blocks). These values are already available point in time via add_data.

Actual Behavior

During universe selection the Similarity object is always null, so report_sentiment.similarity.all and the other similarity fields cannot be read, even though the point in time data type exposes them.

Root Cause

Two things on the universe path drop the data:

  1. DataProcessing/universe.py writes only the 33 sentiment metric columns (datum[3:36]) to the universe file, dropping the trailing similarity columns.
  2. BrainCompanyFilingLanguageMetricsUniverse.Reader calls the metrics only BrainCompanyFilingLanguageMetrics.Parse overload, so Similarity is never populated.

Reproducing the Problem

Add the universe with add_universe(BrainCompanyFilingLanguageMetricsUniverseAll, ...) and read d.ReportSentiment.Similarity.All for any selected datum. It is null, whereas the same field is populated when the symbol is subscribed point in time with add_data(BrainCompanyFilingLanguageMetricsAll, ...).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions