CLI Preprocessor Guide

PallyCon CLI Preprocessor is a command line interface (CLI) tool for watermark preprocessing on an encoded video file.

It runs on a packaging server built by the content service, and processes the source mp4 video into two outputs (0/1 versions, also known as A/B variants). Additional DRM packaging is required for the DASH/HLS stream packaging after the watermark preprocessing.

sequenceDiagram
    participant A as Encoder
    participant B as Packaging server
    participant C as CDN
    A ->> B: Encoded MP4
    Note right of B: CLI Preprocessor
    B -->> B: Watermark Preprocessing
    opt Multi DRM 
    Note right of B: CLI Packager
    B -->> B: Content encryption
    end
    B ->> C: Packaged streams
    Note right of C: DASH / HLS contents<br>(0 and 1 marked)

Modules

You may contact PallyCon team to get the PallyCon CLI Preprocessor after subscribing to PallyCon Forensic Watermarking plan. It contains the below files.

  • FwmPreprocessor: Creates two content files, each marked ‘0’ or ‘1’. For example, when the input content is ‘A.mp4’, it creates ‘A_0.mp4’ and ‘A_1.mp4’. Each of the two generated content can be packaged as DASH / HLS streaming content using PallyCon CLI Packager.

  • fmkeyDemo.dat: This is a demo version key file used for watermark preprocessing in a trial account. In addition to the invisible forensic watermark, a visible watermark (`PALLYCON DEMO') will be added in the video preprocessed with the demo key.

  • Library files: In the lib folder, there are various libraries used in the CLI preprocessor.

Required environments

You need the following environment to run the CLI Preprocessor.

Hardware

AWS GPU instance performance

  • Geforce GTX1050 (Desktop, Pascal) « P2 (Tesla K80) « P3 (Tesla V100), G3 (Tesla M60)
  • When using an AWS EC2 instance as a watermark preprocessing server, we recommend an instance of type P3 or G3 for GPU acceleration.

Software

  • Ubuntu server 18.04, 20.04 or CentOS 7
  • Intel IPP (Integrated Performance Primitives) 2018: Included in the distribution.
  • NVIDIA CUDA Toolkit 9.1 or later (NVIDIA hardware acceleration)
  • Dependency libraries

Installation guide

NVIDIA CUDA Toolkit 10.0

Pallycon CLI Preprocessor supports NVIDIA H/W encoding/decoding. You can check supported GPUs and codecs here.

It supports CUDA Toolkit 9.1, but 10.0 or higher is recommended. You can also install the latest version if there are no compatibility issues with your other programs. If you need to install other CUDA Toolkit version, please refer latest downlaod or archive.

  • Installing on Ubuntu 18.04

    $ distribution=$(. /etc/os-release;echo $ID$VERSION_ID|tr -d '.')
    $ wget https://developer.download.nvidia.com/compute/cuda/repos/${distribution}/x86_64/cuda-repo-${distribution}_10.0.130-1_amd64.deb
    $ sudo dpkg -i cuda-repo-${distribution}_10.0.130-1_amd64.deb
    $ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/${distribution}/x86_64/7fa2af80.pub
    $ sudo apt-get update
    $ sudo apt-get install -y linux-headers-$(uname -r)
    $ sudo apt-get install -y cuda-10-0
    $ rm cuda-repo-${distribution}_10.0.130-1_amd64.deb
    $ sudo reboot
    
  • Installing on CentOS 7

    $ sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    $ wget https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-repo-rhel7-10.0.130-1.x86_64.rpm
    $ sudo rpm -i cuda-repo-rhel7-10.0.130-1.x86_64.rpm
    $ sudo yum clean all
    $ sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
    $ sudo yum install cuda-10-0
    $ rm cuda-repo-rhel7-10.0.130-1.x86_64.rpm
    $ sudo reboot
    
  • Installing on AWS Linux2

    • Please refer to this link
  • Verification

    $ nvidia-smi
    
  • References

Install Dependencies

  • Ubuntu 18.04 version

    $ sudo apt update
    $ sudo apt install -y \
            libva-drm2 \
            libvdpau1 \
            libmediainfo0v5 \
            libpng16-16
    
  • Ubuntu 20.04 version

    $ sudo apt update
    $ sudo apt install -y \
            libnuma1 \
            libva-drm2 \
            libvdpau1 \
            libmediainfo0v5 \
            libpng16-16
    
  • CentOS 7 version

    $ sudo yum update
    $ sudo yum install -y \
            openssl-devel \
            libpng-devel \
            libmediainfo-devel
    

CLI Preprocessor usage

Execute the FwmPreprocessor executable file in the bin folder in the downloaded archive with the following command.

Required parameters

Parameter Description Example Remarks
-i Input media filename -i ~/demo/input.mp4
-o Output media filename -o ~/demo/output.mp4 The resulting output will be two files with ‘0’ and ‘1’ marked. (~/demo/output_0.mp4 and ~/demo/output_1.mp4)
--fmkey_file Filename of seed key --fmkey_file fmkey.dat - Trial customer: Enter fmkeyDemo.dat for trial seed key file included in the distribution.
- Commercial customer: Enter the commercial seed key received separately after subscription.
--access_key Access Key for service authorization --access_key QSTO2zE7 - Trial customer: Enter FOR_INKA_FORENSIC_WATERMARK_TEST
- commercial customer of watermarking service: Enter the access key value shown in the PallyCon Console site
The access key value is used in pairs with the seed key file(fmdat.key) stored in the CLI Preprocessor’s ‘bin’ folder. For watermark trials using the CLI Preprocessor, you must use the access key for the trial (‘FOR_INKA_FORENSIC_WATERMARK_TEST’), not the access key issued at the PallyCon Console site. The seed key file for the trial is included in the deployed CLI Preprocessor. To use the CLI Preprocessor commercially, you need to replace the file with the commercial seed key that is delivered after a separate contract. And you need to enter the value of the access key issued at PallyCon Console site for the --access_key parameter.
  • Descriptions of all parameters can be checked with ‘-h’ option. FwmPreprocessor -h

Example of FwmPreprocessor parameters

  • Example 1: to apply NVIDIA HW acceleration encoder(nvenc), use ‘-n’ option

    FwmPreprocessor -i ~/demo/input.mp4 -o ~/demo/output.mp4 -n  --fmkey_file fmkeyDemo.dat --access_key FOR_INKA_FORENSIC_WATERMARK_TEST
    
  • Example 2: to apply NVIDIA HW acceleration encoder(nvenc) and decoder(cuvid), use both ‘-n’ and ‘-c’ options

    FwmPreprocessor -i ~/demo/input.mp4 -o ~/demo/output.mp4 -n -c --fmkey_file fmkeyDemo.dat --access_key FOR_INKA_FORENSIC_WATERMARK_TEST
    
If an error occurs that is related to the common library path, you must set LD_LIBRARY_PATH in the PATH environment variable. For details, refer to the FwmPreprocessor.sh file included in the archive file.

DRM packaging (optional)

For content that applies both forensic watermarking and DRM, you can package the ‘0’ and ‘1’ version mp4 files generated by FwmPreprocessor using PallyCon Packager. Depending on the service’s target platform, the two versions of watermark content are packaged in DASH or HLS format, and delivered to the user in real time through the embedder module of the CDN.

For details on DRM packaging and watermark mixing, please refer to the following guide documents.

Next