Introduction

DemuxFS is a live filesystem that aids on the analysis of transport streams in digital TV systems.

At the root of the filesystem, directories are created for each table in the transport stream, such as PMT, PAT, NIT, SDTT, EIT, among others. Inside these directories, virtual files map each field described by the MPEG-2 transport stream specification to the given table, such as program information, scrambling information and general options. This concept also extends to tables described by other standards such as SBTVD, ISDB, DVB and ATSC.

The structure of the filesystem is such that it becomes especially useful for analyzing snippets of transport stream files or contents from live broadcasting. It's also useful for implementing media players, given that all required information is provided by the filesystem tree.

Properties

DemuxFS has a few unique properties. Errors are reported as extended attributes to the table members which don't present the expected data. Extended attributes are also applied to tell the user how a specific file is expected to be parsed, such as "sys.format = <string> [<number>]", meaning that a human-readable string preceeds the raw digit sent in the transport stream for the inspected table member. A practical example of a file contents presented that way would be "AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video [0x1b]".

Tables such as PAT have a subdirectory named Programs, populated by symlinks -- one for each program listed in that table. Those symlinks point back to the directories designed for the PMTs and the NIT, stored in DemuxFS' root. So, if a transport stream is announcing a given program but that program is not transmitted in the PMTs, the symlink to that program will be broken, signalling to the user that the transport stream does not conform to the standards.

Packetized Elementary Streams (PES) are elements which represent audio, video and data, such as closed caption. Those are represented in DemuxFS as FIFOs, a class of files which best describes a producer/consumer model. Throughout FIFOs, applications can request PES data and start reproducing their contents in a very straightforward manner, such as invoking mplayer PMT/Streams/0x10/PES for an audio or video stream. The buffering level used by DemuxFS for those FIFOs can be fine tuned using extended attributes.

Just like Packetized Elementary Streams, Elementary Streams (ES) are made available for the same audio, video and data elements. Those are also represented in DemuxFS as FIFOs, and also respond to the same extended attributes commands for fine tuning their buffer size.

Video stream contents can also be verified through still picture snapshots. As soon as an image viewer application (e.g: the Gimp) opens the snapshot file, DemuxFS collects video PES packets and feeds the FFMpeg library, which decodes those buffers in software and returns back a raster snapshot of the current video frame.

Examples

This is how the root of the filesystem looks like:

	~] cd /Mount/DemuxFS
	/Mount/DemuxFS] l
	31/12 21:00 r-xr-xr-x        0 AIT
	31/12 21:00 r-xr-xr-x        0 DDB
	31/12 21:00 r-xr-xr-x        0 DII
	31/12 21:00 r-xr-xr-x        0 DSI
	31/12 21:00 r-xr-xr-x        0 DSM-CC
	31/12 21:00 r-xr-xr-x        0 NIT
	31/12 21:00 r-xr-xr-x        0 PAT
	31/12 21:00 r-xr-xr-x        0 PMT
	31/12 21:00 r-xr-xr-x        0 SDT
	31/12 21:00 r-xr-xr-x        0 Streams
	31/12 21:00 r-xr-xr-x        0 TOT

Every table is represented by a directory, and inside that directory there's one more directory for each version found in the transport stream. A symlink named Current points to the currently active version:

	/Mount/DemuxFS] l PAT
	31/12 21:00 rwxrwxrwx        0 Current -> Version_1
	31/12 21:00 r-xr-xr-x        0 Version_1

Every table carries a few common entries, found in the common PSI headers. Those are stored inside the versioned directory as standard files, as shown below. The PAT table, which announces every program found in the transport stream, also includes an extra directory named Programs where they are arranged. Reading an entry's extended attributes and its contents is also straightforward using standard tools.

	/Mount/DemuxFS] l PAT/Current
	31/12 21:00 r-xr-xr-x        0 Programs
	31/12 21:00 r-xr-xr-x       33 tableHeader
	
	/Mount/DemuxFS] l PAT/Current/tableHeader
	31/12 21:00 r--r--r--        4 current_next_indicator
	31/12 21:00 r--r--r--        5 identifier
	31/12 21:00 r--r--r--        4 last_section_number
	31/12 21:00 r--r--r--        4 section_length
	31/12 21:00 r--r--r--        4 section_number
	31/12 21:00 r--r--r--        4 section_syntax_indicator
	31/12 21:00 r--r--r--        4 table_id
	31/12 21:00 r--r--r--        4 version_number
	
	/Mount/DemuxFS] getfattr -d -m - PMT/0x20/Current/tableHeader/table_id
	# file: PMT/0x20/Current/table_id
	system.format="number"
	
	/Mount/DemuxFS] cat PMT/0x20/Current/tableHeader/table_id
	0x02

Each program listed in the PAT points back to the entries in the NIT and/or in the PMT. Again, symlinks are used to organize the hierarchy:

	/Mount/DemuxFS] l PAT/Current/Programs
	31/12 21:00 rwxrwxrwx        0 0000 -> ../../../NIT/Current
	31/12 21:00 rwxrwxrwx        0 0x20 -> ../../../PMT/0x20/Current

	/Mount/DemuxFS] l PMT/0x20/Current
	31/12 21:00 r-xr-xr-x        0 AudioStreams
	31/12 21:00 r-xr-xr-x        0 PARENTAL_RATING_0
	31/12 21:00 r-xr-xr-x        0 VideoStreams
	31/12 21:00 r-xr-xr-x       33 tableHeader
	31/12 21:00 r--r--r--        6 pcr_pid
	31/12 21:00 r--r--r--        4 program_information_length

The video stream directories, as stated above, offer direct access to their PES and ES streams, just like its current snapshot, as a still picture file. Decoding information is given in the file stream_type_identifier, named after the PMT table member.

	/Mount/DemuxFS] l PMT/0x20/Current/VideoStreams
	31/12 21:00 rwxrwxrwx        0 Primary -> 0x111
	31/12 21:00 r-xr-xr-x        0 0x111

	/Mount/DemuxFS] l PMT/0x20/Current/VideoStreams/Primary
	31/12 21:00 rwxrwxrwx        0 ES
	31/12 21:00 rwxrwxrwx        0 PES
	31/12 21:00 r-xr-xr-x        0 STREAM_IDENTIFIER
	31/12 21:00 r--r--r--        5 elementary_stream_pid
	31/12 21:00 r--r--r--        4 es_information_length
	31/12 21:00 r--r--r-- 16777215 snapshot.png
	31/12 21:00 r--r--r--       79 stream_type_identifier

	/Mount/DemuxFS] cat PMT/0x20/Current/VideoStreams/Primary/stream_type_identifier
	AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video [0x1b]

DemuxFS also handles the protocol stack of DSM-CC, in which case both AIT, DII, DSI and DDB tables are exported to the filesystem. The DDB payload is decoded and exported to the filesystem, too, allowing for the inspection of interactive applications.

	/Mount/DemuxFS] l DSI/0x384/Current
	31/12 21:00 rwxrwxrwx        0 DII -> ../../../DII/0x384/Current
	31/12 21:00 r-xr-xr-x       23 biopServiceGatewayInformation
	31/12 21:00 r-xr-xr-x       81 dsmccMessageHeader
	31/12 21:00 r-xr-xr-x       32 tableHeader
	31/12 21:00 r--r--r--        4 compatibility_descriptor_length
	31/12 21:00 r--r--r--        4 private_data_length
	31/12 21:00 r--r--r--       20 server_id

	/Mount/DemuxFS] l DII/0x384/Current
	31/12 21:00 r-xr-xr-x       82 dsmccMessageHeader
	31/12 21:00 r-xr-xr-x       18 module_01
	31/12 21:00 r-xr-xr-x       19 module_02
	31/12 21:00 r-xr-xr-x       18 module_03
	31/12 21:00 r-xr-xr-x       18 module_04
	31/12 21:00 r-xr-xr-x       18 module_05
	31/12 21:00 r-xr-xr-x       18 module_06
	31/12 21:00 r-xr-xr-x       33 tableHeader
	31/12 21:00 r--r--r--        4 ack_period
	31/12 21:00 r--r--r--        5 block_size
	31/12 21:00 r--r--r--        4 download_id
	31/12 21:00 r--r--r--        4 number_of_modules
	31/12 21:00 r--r--r--        4 private_data_length
	31/12 21:00 r--r--r--        4 t_c_download_scenario
	31/12 21:00 r--r--r--        4 t_c_download_window
	31/12 21:00 r--r--r--        4 window_size
	
	/Mount/DemuxFS] l DDB/0x384/Current
	31/12 21:00 r-xr-xr-x       18 module_01
	31/12 21:00 r-xr-xr-x       19 module_02
	31/12 21:00 r-xr-xr-x       18 module_03
	31/12 21:00 r-xr-xr-x       18 module_04
	31/12 21:00 r-xr-xr-x       18 module_05
	31/12 21:00 r-xr-xr-x       18 module_06

	/Mount/DemuxFS] l DSM-CC
	31/12 21:00 r-xr-xr-x        0 Test Program
			
	/Mount/DemuxFS] l DSM-CC/Test\ Program
	31/12 21:00 r-xr-xr-x        0 BUG_TST_20090430_01
					
	/Mount/DemuxFS] l DSM-CC/Test\ Program/BUG_TST_20090430_01
	31/12 21:00 r--r--r--     4996 ECMAScriptPOC.html
	31/12 21:00 r--r--r--     9627 logo.png
	31/12 21:00 r--r--r--     1618 main.ncl

Downloading

DemuxFS can be downloaded from Subversion. Please use the following commands to check it out:

	svn co https://demuxfs.svn.sourceforge.net/svnroot/demuxfs demuxfs

Alternatively, you may want to download a GNU tarball of the current snapshot.

Licensing

DemuxFS is licensed under the BSD license.

About

DemuxFS was developed by Lucas C. Villa Real as a case study for a MSc dissertation at the University of São Paulo. It's currently aiming full compatibility with the Brazilian Digital TV System (SBTVD), which shares a lot with ISDB and has many of the features also seen in DVB and ATSC.