QN Subtitles

[dmtrjsg]

Context

Design Epic link
Dev Epic

Scope

See below 👇

[kdembler]

Here's my proposal for the subtitles spec:

Desired QN schema

We should extend the QN schema for Video entity like so:

type VideoSubtitle {
    # globally unique ID
    id: ID!

    video: Video!

    # Atlas will use 'subtitles' | 'closed-captions' for now and possible other types in the future.
    # I think this can be an opaque string to QN for the time being, don't think we will need any additional fields per variant.
    type: String!

    # Language may not make sense for all subtitle types in the future, but I think it's easier to just place this flat on VideoSubtitle instead of adding variants and nested fields
    language: Language!

    # MIME type description of format used for this subtitle
    mimeType: String!

    # Storage object representing the subtitle file
    asset: StorageDataObject
}

type Video {
    # ...

    subtitles: [VideoSubtitle!]!
}

Setting/changing subtitles

I think we should use existing runtime capabilities of providing assets via the Video object. We can update the VideoMetadata protobuf message to include subtitles data. There is an issue that subtitles needs to be a list, but how do we handle updates to that list? I suggest to always overwrite the full list. QN can use (VideoSubtitle.type, VideoSubtitle.language) pair as a subtitle identifier. Then each subtitles object in protobuf message can potentially reference a new asset to upload that can overwrite the old one if necessary. I don't have much Protobuf experience so take this with a grain of salt:

message VideoMetadata {
    // ...
    repeated VideoSubtitle subtitles = 17;
}

message VideoSubtitle {
  required string type = 1;

  // index into external [assets array](#.Assets)
  optional uint32 new_asset = 2;

  // ISO_639-1 Language [Code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)
  required string language = 3;

  required string mimeType = 4;
}

With this we handle subtitles with regular update_video extrinsic. No runtime changes required and most likely minimal changes to QN needed

[bedeho]

This seems very sound, just to confirm, each subtitle object is a separate data object, correct? so if the viewer wants to add a new subitile in the future, they just update a new bundle, right? they dont need republish one new bundle with all subtitles again?

There is an issue that subtitles needs to be a list, but how do we handle updates to that list? I suggest to always overwrite the full list.

I did not understand this part.

With this we handle subtitles with regular update_video extrinsic. No runtime changes required and most likely minimal changes to QN needed

Yep, looks good sir 😎 . I think its very promising that we are able to do it this way, really shows that we have a sound way to unlock very diverse product features without devs needing to ask for permission.

[kdembler]

Yes, each subtitle would be a separate object, correct.

There is an issue that subtitles needs to be a list, but how do we handle updates to that list? I suggest to always overwrite the full list.

I did not understand this part.

Each video has to have a list of subtitles. So the question I raised is how we provide this list or update it. One approach would be to only include new subtitles in the protobuf message with video metadata. This way, however, it's hard to indicate that we want to remove a specific subtitle. So my suggestion to alleviate that is make sure that every time video gets updated, this update needs to mention the full set of subtitles that this video should have, basically overwriting whatever was known before. Does that help?

[dmtrjsg]

Needs to be rebased for Carthage