SDL - Sequence Description Language

From Kaltura Wiki

A redefinition to SMIL - draft 1: SDL_to_SMIL

[edit] Background

Kaltura CVE (Collaborative Video Editing SDK) is a set of tools to parse and build a Kaltura Video - a video sequence of images, videos, sounds, transitions, effects, etc.
The core foundation of the Kaltura Video is it's SDL. The SDL is language that describe the flow of the Kaltura Video according to an XML schema (no XSL available at the moment).
A Kaltura Video is described using SDL.

[edit] SDL from a bird's eye view

<xml>
	<MetaData>
		<Publish>
			<Application>'Keditor'</Application>
		</Publish>
		<SeqDuration></SeqDuration>
		<ShowVersion></ShowVersion>
	</MetaData>
	<VideoAssets>
		<EndTransition />
	</VideoAssets>
	<AudioAssets>
		<EndTransition />
	</AudioAssets>
	<VoiceAssets>
		<EndTransition />
	</VoiceAssets>
	<AudioEffects>
		<EndTransition />
	</AudioEffects>
	<Plugins>
		<Overlays></Overlays>
		<Effects></Effects>
	</Plugins>
</xml>

[edit] MetaData section

<MetaData> contains the global definitions of the SDL;
To identify the publisher tool used to create the SDL <Application>,
The total duration of the Kaltura Video <SeqDuration>{duration in seconds}</SeqDuration> - this is the duration of the longest Assets section,
The version of the edit <ShowVersion>,
The url for the thumbnail of the Kaltura Video <ThumbUrl>,
The hosting server <Host>, Information about server processing <Pending>,<LastPendingTimeStamp> and more...
Event though the <MetaData> section is not mandatory to process a Kaltura Video, the <SeqDuration> is mandatory and should be always included.

[edit] Assets sections

Following the MetaData, the 4 basic data sections that are used to produce a Kaltura Video –

1. The <VideoAsset> represents the visual elements in a video; Video, Image and Solid Color, the transitions between following assets and effects attached on the those assets.
   
   1. <EndTransition> represents any transition used between two assets; this element is positioned in the SDL inside the asset tag itself to improve SDL readability.
   2. <AssetEffect> represents any effect applied to the asset; this element is positioned in the SDL inside the asset tag itself to improve SDL readability.
2. The <AudioAsset> represents the master sound track of the video.
   
3. The <VoiceAsset> represents the voice over channel for creating dubbings or synching voice messages.
   
4. The <AudioEffects> represents an extra channel for audio effects.
   
   

[edit] Plugins section

The <Plugins> section of the SDL describe all the plugins used in the Kaltura Video. Plugins represents all visual or audio effects;

1. <Overlays> from subtitles and text on top the video thorough adding extra channels of video or images and using masks or chroma keying.
   
2. The <Effects> represents all transformations and manipulation applied to the sequence (ie. Black & White, Invert Colors…).
   
   

[edit] The Asset Element

Each Assets (plural - VideoAssets, AudioAssets...) element is compound from it's corresponding Asset (single) element.
This element is almost identical across all Asset elements, but can be slightly vary in some aspects.
The basic Asset element is defined as follows:

<___Asset k_id={entry id on kaltura server} type={VIDEO | AUDIO | SOLID | IMAGE | SILENCE | VOICE} name={name of entry on kaltura server} url={url of file}>
	<StreamInfo file_name={file name} posX="0" posY="0" start_byte="-1" end_byte="-1" total_bytes="-1" real_seek_time="-1" 
		start_time={0-maximum length of original file} len_time={length in seconds} 
		volume={the volume specific to the asset} pan={the audio balance specific to the asset} 
		Clipped_Start={the time we cut the asset relative to the original stream} Clipped_Len={the duration we cut the asset relative to the original stream} />
	<EndTransition cross={1 - this asset and the following will overlap for the specified length} 
			type={type of transition - the id of the plugin on kaltura server} 
			StartTime={the time in seconds relative to the asset duration on which the transition should start} 
			length={the duration of the transition - in seconds}>
		{transition plugin specific values and description}
	</EndTransition>
	<AssetEffect type={type of effect - the id of the plugin on kaltura server} 
			StartTime={the time in seconds relative to the asset duration on which the effect should start} 
			length={the duration of the effect - in seconds}>
		{effect plugin specific values and description}
	</AssetEffect>
	<VolumePoints>
		<VolumePoint time={0-length} volume={0-1} />
	</VolumePoints>
</___Asset>

[edit] Special asset types

1. SOLID - the solid asset type describe an empty color slide of a selected color. this asset is generated at runtime. the value of its color is saved in the "name" attribute in the SDL.
2. SILENCE - due to the SDL's linear nature we use a place holder of type SILENCE to make AUDIO assets act as non linear and have silent parts in the Kaltura Video.

[edit] Available Asset types

1. vidAsset - to describe visual elements of types; VIDEO, SOLID and IMAGE.
2. AudAsset - to describe sound elements of types; AUDIO, SILENCE (as a place holder for no audio segment) and VOICE.

(* due to case sensitivity bug in SDL definition the vidAsset is lower case v and AudAsset is upper case A). 
(* Although defined in SDL, AssetEffect is still not present in current version).

[edit] SDL flow

[edit] Assets section

The Assets section of the SDL is linear in time, each asset should be defined in it's right position according to its time on the sequence,
and it's start_time is defined relative to the previous Asset in section.

[edit] Plugins section

The Plugins section of the SDL is not linear, each plugin satrt_time is defined according to the global sequence time, thus can have value of 0 - <SeqDuration>.

[edit] The Plugins Framework

[edit] Overview

Based on Adobe® Flex’s® modules framework, Keditor and Kplayer can be easily extended. Minimum requirements for writing new plugins (i.e. transitions, overlays, or effects) include basic knowledge of ActionScript and the Flex SDK & Compiler.

Plugins get access to the media library, and all the elements of the Kaltura Video sequence, such as the main timeline assets, the audio assets and the overall volume control. Plugins can also load their own data from external sources, show visual elements on-top of the main timeline and play their own audio or video assets.

Plugins exist on the timeline for a user-defined duration of time.
For more details on the plugin framework: Keditor_kplayer#The_Plugins_Framework

[edit] SDL definition for plugins

In addition to the transitions and asset applied effects that can be defined inside the Asset element in the Assets sections, there is another type of plugin asset.
This section of the SDL is not linear and is used to define plugins such as overlays and effects that are not attached to a single asset time, these assets use a global sequence time.

<Plugins>
	<Overlays>
	</Overlays>
	<Effects>
	</Effects>
</Plugins>

Inside the relative section (Overlays for OVERLAY types and Effects for EFFECT types), the plugin is defined as follows:

<Plugin type={type of plugin on the kaltura server} StartTime={the global sequence time in seconds at which plugin starts to affect} 
                length={the duration of the plugin in seconds}>
	{specific plugin elements (plugin.toXML())}
</Plugin>

Every plugin has its own extra properties to be saved.
The plugins in the CVE implements a toXML() function.
That function is used inside the <Plugin></Plugin> element to save those values and instantiate later when parsing the SDL.