Support SSA/ASS styling

[jakobkukla]

Add support for SSA/ASS styling. Currently there is only "minimal" support for ASS subtitles since styling was removed from the initial PR. (See #2281)

It would be great to have full support of the spec. Many subtitles use these styling features to work directly with the video (e.g. to substitute text in a video). SSA/ASS styling is also very common in certain genres of media like Cartoons, Anime or Karaoke.

[andrewlewis]

@icbaker Tentatively marking as an enhancement, but please dupe or close as needed. Thanks!

[icbaker]

Thanks for the request.

Unfortunately we don't currently have any capacity to improve ExoPlayer's SSA/ASS styling support. However we would welcome high quality PRs in this area.

I'm also happy to discuss various approaches with PR authors if a new feature doesn't fit easily into the existing code structure (either before the PR, or in the review, whichever is easier).

[13steinj]

Could we determine a basic checklist of what isn't supported, to determine what features of the spec can be focused on in order of size and/or ease of implementation?

(I'm not familiar with what exactly even is supported currently).

[icbaker]

Could we determine a basic checklist of what isn't supported, to determine what features of the spec can be focused on in order of size and/or ease of implementation?

That sounds like a good idea - the best way to do that is probably to compare the SsaDecoder to a copy of the spec. I've found the SSA/ASS spec to be fairly loose, and I'm not sure there's a canonical definition, but I've generally referred to this when working with these files in the past: http://moodub.free.fr/video/ass-specs.doc (linked from Matroska's page on SSA).

In case it's helpful, we would probably prioritise things roughly in this order:

• Features that look like they're implemented but don't work properly (basically bugs).
• Features that are similar to existing ones, and so can be implemented with small incremental changes.
• Features that don't fit into the existing code architecture - so implementing them requires significant refactoring.

I don't know if SSA has anything in the final category - but if it looks like it does then please come talk to us before spending ages refactoring SsaDecoder, as we might have some ideas about the best way to approach the problem.

[cosmo321]

I'm not a developer, so I can't exactly contribute a lot, but proper support of ass/ssa in Exoplayer is something I'd very much would like to have, as Plex for Android currently makes me sad...
I don't know if this makes sense in the context of Exoplayer, but it seems to me that the closest thing we have to a "standard" for ass/ssa is with libass, as a lot of notable players have implemented it and it is actively developed. I think at least being compliant with how libass renders subtitles would be a good idea.

[icbaker]

I don't know if this makes sense in the context of Exoplayer, but it seems to me that the closest thing we have to a "standard" for ass/ssa is with libass, as a lot of notable players have implemented it and it is actively developed. I think at least being compliant with how libass renders subtitles would be a good idea.

Thanks for the pointer to libass, I took a quick look. I think you're possibly suggesting two alternative things:

• ExoPlayer could depend on libass and use it's logic for all the SSA rendering.
• People working on improving ExoPlayer's existing SSA rendering logic could use libass as a reference implementation to see how different pieces of content should be rendered, as an augmentation to the "spec".

In case anyone is considering trying the first of these, I'd like to pre-emptively warn against it. libass looks like it's written in C, and we're very reluctant to pull in native/JNI dependencies unless a pure Java implementation of the same behaviour is deemed ~impossible. Given SSA/ASS is a relatively simple plaintext-based format, it seems completely feasible to parse it fully using pure Java - so it's extremely unlikely we'd accept a PR that introduced a native dependency to achieve this. There's more discussion about (not) using native deps for parsing subtitles in #7746 and #7749.

The second suggestion (use it as a reference implementation) makes perfect sense - and afaict it's used by VLC, so it should be enough to just play any content with VLC and compare to ExoPlayer's behaviour.

[cosmo321]

I leave that up to actual developers to decide. I just thought I'd point out that libass exists and is probably the best reference we have, and that rendering subs differently from libass could potentially make people like me wonder why the subs behave differently in ExoPlayer. The second alternative is probably the better option for ExoPlayer. :)

[szaboa]

hey @icbaker , I'll check what the current SsaDecoder supports and what is missing in regards of styling, based on the specs you linked above, then I could probably start and implement those step by step (with my limited time).

@jakobkukla did you have anything specific in mind? I mean is there a specific styling option which you would like to see first?

[icbaker]

Thanks for taking a look! Breaking down the work into small PRs definitely sounds good.