ffmpeg

ffmpeg: audio visualization tricks

September 2, 2017

Often one wants to share audio online, but it seems like video as a format has many more options: mastodon, twitter, facebook, youtube, all allow to upload video, but not only audio. Here are some ffmpeg tricks, how to add interesting video to your audio file, often autogenerated visuals. All the code is supposed to be used as one line without line-breaks.

Audio Vector Scope

ffmpeg -i INPUT_AUDIO.wav -filter_complex 
"[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v]; 
 [v]pad=854:480:187:0[out]" 
 -map "[out]" -map 0:a 
-b:v 700k -b:a 360k 
OUTPUT_VIDEO.mp4

ffmpeg -i INPUT_AUDIO.wav -filter_complex

"[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v];

[v]pad=854:480:187:0[out]"

-map "[out]" -map 0:a

-b:v 700k -b:a 360k

OUTPUT_VIDEO.mp4

The code above creates a mp4 video file with a vectorscope nicely centered inside a 854×480 (480p) video. If you need a 1:1 video, just exclude the pad part:

ffmpeg -i INPUT_AUDIO.wav -filter_complex 
"[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v]" 
 -map "[v]" -map 0:a 
-b:v 700k -b:a 360k 
OUTPUT_VIDEO.mp4

ffmpeg -i INPUT_AUDIO.wav -filter_complex

"[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v]"

-map "[v]" -map 0:a

-b:v 700k -b:a 360k

OUTPUT_VIDEO.mp4

Documentation on ‘avectorscope’ filter is here: https://ffmpeg.org/ffmpeg-filters.html#avectorscope. One can play with zoom and other options to produce desired form.

Show waves

ffmpeg -i INPUT.wav -filter_complex 
"[0:a]showwaves=mode=line:s=hd480:colors=White[v]" 
-map "[v]" -map 0:a -pix_fmt yuv420p 
-b:a 360k -r:a 44100  OUTPUT.mp4

ffmpeg -i INPUT.wav -filter_complex

"[0:a]showwaves=mode=line:s=hd480:colors=White[v]"

-map "[v]" -map 0:a -pix_fmt yuv420p

-b:a 360k -r:a 44100 OUTPUT.mp4

more options: http://www.ffmpeg.org/ffmpeg-filters.html#showwaves

Showspectrum

ffmpeg -i INPUT.wav  -filter_complex 
"[0:a]showspectrum=s=854x480:mode=combined:slide=scroll:saturation=0.2:scale=log,format=yuv420p[v]" 
 -map "[v]" -map 0:a 
 -b:v 700k -b:a 360k OUTPUT.mp4

ffmpeg -i INPUT.wav -filter_complex

"[0:a]showspectrum=s=854x480:mode=combined:slide=scroll:saturation=0.2:scale=log,format=yuv420p[v]"

-map "[v]" -map 0:a

-b:v 700k -b:a 360k OUTPUT.mp4

Above code will create almost completely desaturated spectrum of the audio sliding from right to left. Again, there are various options to tweak, see here: https://ffmpeg.org/ffmpeg-filters.html#showspectrum-1

Histogram

ffmpeg -i INPUT.wav -filter_complex 
"[0:a]ahistogram=s=hd480:slide=scroll:scale=log,format=yuv420p[v]" 
 -map "[v]" -map 0:a 
 -b:a 360k OUTPUT.mp4

ffmpeg -i INPUT.wav -filter_complex

"[0:a]ahistogram=s=hd480:slide=scroll:scale=log,format=yuv420p[v]"

-map "[v]" -map 0:a

-b:a 360k OUTPUT.mp4

Documentation: https://ffmpeg.org/ffmpeg-filters.html#ahistogram

Static spectrogram

Sometimes you want to just create a static image.

# create a spectrogram as a single frame
ffmpeg -i INPUT.wav -lavfi 
showspectrumpic=s=hd480:legend=0,format=yuv420p 
SPECTROGRAM.png

# add png to audio - you need to know the length of audio
ffmpeg -loop 1 -i SPECTROGRAM.png -i INPUT.wav 
-s hd480 -t 00:01:00 -pix_fmt yuv420p 
-b:a 360k -r:a 44100 OUTPUT.mp4

# create a spectrogram as a single frame

ffmpeg -i INPUT.wav -lavfi

showspectrumpic=s=hd480:legend=0,format=yuv420p

SPECTROGRAM.png

# add png to audio - you need to know the length of audio

ffmpeg -loop 1 -i SPECTROGRAM.png -i INPUT.wav

-s hd480 -t 00:01:00 -pix_fmt yuv420p

-b:a 360k -r:a 44100 OUTPUT.mp4

Above one is in two steps. More info here: http://www.ffmpeg.org/ffmpeg-filters.html#showspectrumpic

Drawtext

ffmpeg -i INPUT.wav -filter_complex 
"[0:a]showwaves=mode=line:s=hd480:colors=Yellow@0.5|Blue@0.5:scale=sqrt[v];
 [v]drawtext=text='Rudi Rudi - Still Standing':fontcolor=White@0.5:
 fontsize=30:font=Arvo:x=(w-text_w)/5:y=(h-text_h)/5[out]" 
 -map "[out]" -map 0:a -pix_fmt yuv420p 
 -b:a 360k -r:a 44100  OUTPUT.mp4

ffmpeg -i INPUT.wav -filter_complex

"[0:a]showwaves=mode=line:s=hd480:colors=Yellow@0.5|Blue@0.5:scale=sqrt[v];

[v]drawtext=text='Rudi Rudi - Still Standing':fontcolor=White@0.5:

fontsize=30:font=Arvo:x=(w-text_w)/5:y=(h-text_h)/5[out]"

-map "[out]" -map 0:a -pix_fmt yuv420p

-b:a 360k -r:a 44100 OUTPUT.mp4

Add text to any of the above with a “drawtext” filter. More options here: http://www.ffmpeg.org/ffmpeg-filters.html#drawtext-1

jpeg-LS glitching with FFmpeg

October 26, 2016

In an older version of FFmpeg it was possible to glitch the image with JPEG-LS codec. Newer versions of FFmpeg don’t work anymore in this way, so one must download an old version and compile it (keep it local) – here 2.0.7 is used.

This script takes a video file as an argument, extracts frames, glitches them, and gathers frames back into a video file.

#!/bin/bash

# jpeg-LS glitcher by nova@deviator.si http://deviator.si

# check existence of input argument
# the $# variable will tell you the
# number of input arguments the script was passed. 
if [ $# -eq 0 ]
  then
    echo "Error. No arguments supplied. Please define a video"
    exit 1
fi

# does the movie file actually exist?
if [ ! -f $1 ]; then
    echo "Error. File not found!"
    exit 1
fi

# get this session's "ID"
SID=`date +%Y%m%d_%H%M%S`

# jpegls glitch needs an older ffmpeg 
FFMPEG=/home/random/src/ffmpeg/ffmpeg-2.0.7/ffmpeg

# create folder for this session
DIR=jpegls_$SID
echo "+++ creating folder $DIR ..."
mkdir $DIR

# create tmp folder
echo "+++ creating folder $DIR/tmp/ ..."
mkdir $DIR/tmp/

echo "+++ extracting all frames as images..."
echo
echo "----------------------------------------------------------"
echo
ffmpeg -loglevel 16 -i $1 $DIR/tmp/frame%5d.jpg

echo 
echo "----------------------------------------------------------"
echo "+++ processing each image..."

for i in $DIR/tmp/*.jpg
do
    
    echo "processing $i "
    $FFMPEG -loglevel 16 -i $i -pred 80 -c:v jpegls $i.jpgls.jpg
    ffmpeg -loglevel 16 -i $i.jpgls.jpg -c:v png $i.png
    
done

echo
echo "----------------------------------------------------------"
echo "+++ putting frames back into a movie ... "
ffmpeg -loglevel 16 -i $DIR/tmp/frame%5d.jpg.png $DIR/$SID.$1

echo
echo "+++ removing temporary folder ..."
rm -rf $DIR/tmp/

echo "\n--- end of the program. "

#!/bin/bash

# jpeg-LS glitcher by nova@deviator.si http://deviator.si

# check existence of input argument

# the $# variable will tell you the

# number of input arguments the script was passed.

if [ $# -eq 0 ]

then

echo "Error. No arguments supplied. Please define a video"

exit 1

# does the movie file actually exist?

if [ ! -f $1 ]; then

echo "Error. File not found!"

exit 1

# get this session's "ID"

SID=`date +%Y%m%d_%H%M%S`

# jpegls glitch needs an older ffmpeg

FFMPEG=/home/random/src/ffmpeg/ffmpeg-2.0.7/ffmpeg

# create folder for this session

DIR=jpegls_$SID

echo "+++ creating folder $DIR ..."

mkdir $DIR

# create tmp folder

echo "+++ creating folder $DIR/tmp/ ..."

mkdir $DIR/tmp/

echo "+++ extracting all frames as images..."

echo

echo "----------------------------------------------------------"

echo

ffmpeg -loglevel 16 -i $1 $DIR/tmp/frame%5d.jpg

echo

echo "----------------------------------------------------------"

echo "+++ processing each image..."

for i in $DIR/tmp/*.jpg

echo "processing $i "

$FFMPEG -loglevel 16 -i $i -pred 80 -c:v jpegls $i.jpgls.jpg

ffmpeg -loglevel 16 -i $i.jpgls.jpg -c:v png $i.png

done

echo

echo "----------------------------------------------------------"

echo "+++ putting frames back into a movie ... "

ffmpeg -loglevel 16 -i $DIR/tmp/frame%5d.jpg.png $DIR/$SID.$1

echo

echo "+++ removing temporary folder ..."

rm -rf $DIR/tmp/

echo "\n--- end of the program. "

timelapse aka speeding up with ffmpeg

May 9, 2016

These days I’m recording my work in the studio using a timelapse function in my Panasonic TM700 HD camera. So it happened that I forgot to turn on the the function (needs to be turned on everytime you start recording) which usually records one frame every 10 seconds, so I ended up with a “normal” recording and wanted to convert that to what camera usually does. Ffmpeg to the rescue! To do some frame manipulation a video filter “setpts” is what one needs. Gathering knowledge online with help of two pages:

http://blog.grio.com/2012/01/fast-and-slow-motion-video-with-ffmpeg.html
https://trac.ffmpeg.org/wiki/How to speed up / slow down a video

So, supposedly, the right way to change video speed using ffmpeg is by adjusting the “presentation time stamp” (PTS). This adjusts frames’ metadata related to how long each is displayed—exactly what you want.

this is the crucial piece of code that needs to be passed to ffmpeg:
-filter:v "setpts=2.0*PTS"

Or a more practical example using fraction – if original framerate is 25 frames per second I only need evert 250th one:
$ ffmpeg -i INPUTFILE.mkv -filter:v "setpts=(1/250)*PTS" OUTPUTFILE.mkv

My final conversion was from full-HD .mts to 720p .webm and to mp4:
$ ffmpeg -i INPUT.mts -filter:v "setpts=(1/250)*PTS" -s 1280x720 -c:v libvpx -crf 5 -b:v 8M -an OUTPUT.webm $ ffmpeg -i INPUT.mts -filter:v "setpts=(1/250)*PTS" -s 1280x720 -c:v libx264 -preset slow -crf 10 -an OUTPUT.mp4

Prepare audio for YouTube with avconv

June 10, 2014

How to create a video file from a single image and audio track using avconv:

avconv -loop 1 -i IMAGE.jpg \
       -i AUDIO.wav \
       -c:v libx264 -tune stillimage -pix_fmt yuv420p -s 854x480 \
       -c:a aac -strict experimental -b:a 192k -shortest OUTPUT.mp4

avconv -loop 1 -i IMAGE.jpg \

-i AUDIO.wav \

-c:v libx264 -tune stillimage -pix_fmt yuv420p -s 854x480 \

-c:a aac -strict experimental -b:a 192k -shortest OUTPUT.mp4

Thanks to a question/recipe from SuperUser:
superuser.com/questions/571100/avconv-video-from-single-jpg-and-ogg

sound, noise, music & new media

ffmpeg

ffmpeg: audio visualization tricks

Audio Vector Scope

Show waves

Showspectrum

Histogram

Static spectrogram

Drawtext

jpeg-LS glitching with FFmpeg

timelapse aka speeding up with ffmpeg

Prepare audio for YouTube with avconv

About

Newsletter

Events