---
title: 'Finding the dull parts of a screen recording'
canonical: 'https://www.dealmobile.co.uk/writing/finding-the-dull-parts-of-a-screen-recording'
url: 'https://www.dealmobile.co.uk/writing/finding-the-dull-parts-of-a-screen-recording'
type: Post
published: '2026-10-21'
summary: 'How Reelify scores each frame of a screen recording by how much it changed, turns the scores into cuts with a margin around them, works out a speed to fit a target length and cuts out each segment, checked against a test clip with known content.'
---

# Finding the dull parts of a screen recording

Published 2026-10-21

![A chart of motion scores across a 30 second test clip: flat near zero, apart from one tall block from about 15 to 20 seconds that is highlighted as the part kept, above a dashed threshold line.](https://www.dealmobile.co.uk/sites/default/files/styles/almanac_shot_md/public/content/images/posts/finding-the-dull-parts-of-a-screen-recording/hero.png.webp?itok=fKT3YsZ-)

A long screen recording has plenty of stretches where nothing happens on screen. [Reelify](https://www.dealmobile.co.uk/projects/reelify.md) is an experiment in cutting them out automatically, on my own machine. It scores each sampled frame by how much it differs from the one before, cuts the stretches where nothing changes, and can speed up what’s left to fit a target length. Most of its history is from one day, 25 April 2026, so there isn’t much code, and most of the interesting decisions are in a few small files.

## Scoring a frame

`src/reelify/analyser.py` asks FFmpeg for small greyscale frames rather than decoding them in Python:

```python
cmd = [
    "ffmpeg", "-i", str(video_path),
    "-vf", f"fps={sample_fps},scale={_THUMB_WIDTH}:{thumb_height}",
    "-f", "rawvideo", "-pix_fmt", "gray",
    "-",
]

```

`sample_fps` is a quarter of the recording’s frame rate, and `_THUMB_WIDTH` is 320, so a 1080p recording at 30 frames a second becomes 7.5 frames a second at 320 by 180. Each frame arrives on standard output as 57,600 bytes, one per pixel, with no header. Python reads exactly that many bytes at a time and compares each thumbnail with the one before:

```python
gray = np.frombuffer(raw, dtype=np.uint8).reshape((thumb_height, _THUMB_WIDTH))
if prev_gray is None:
    scores.append(0.0)
else:
    mad = np.mean(np.abs(gray.astype(np.float32) - prev_gray.astype(np.float32)))
    scores.append(mad / 255.0)

```

The score is the mean absolute difference, scaled to between 0 and 1. Zero means the two thumbnails are identical, and 1 would mean every pixel went from black to white or back.

An earlier version read the video through OpenCV at full size, reading one frame in four and calling `grab()` to skip the rest. FFmpeg still decodes every frame, but now it drops three in four and shrinks the rest before Python sees them, so Python handles 57,600 bytes per sample instead of a full-size colour frame.

## What the numbers look like

To see what a score means, I made a test clip: frames drawn with NumPy and encoded with FFmpeg, 30 seconds at 1920 by 1080 and 30 frames a second. It shows a white page with rows of dark blocks, like lines of text. Nothing moves for the first 10 seconds. Then a 16 by 24 pixel pointer moves across the page for 5 seconds, the page scrolls 6 pixels a frame for 5 seconds, and nothing moves for the last 10.

The pointer scored about 0.0002. The scroll scored about 0.43. The still parts scored 0, apart from two blips of about 0.001 near 8.33 and 25 seconds, the two points in the still parts where the encoder had started a new keyframe, every 250 frames.

The default threshold in `reelify process` is 0.02, which the project’s architecture notes say mirrors auto-editor’s motion threshold. At that level, a pointer moving on its own counts as idle and gets cut, and a scroll is more than twenty times over. The `cli` preset lowers the threshold to 0.005, which still leaves the pointer out. Going much lower starts catching the encoder: at 0.0001, each keyframe blip became a short active stretch of its own.

## Padding the cuts

`classify()` in `src/reelify/classifier.py` turns scores into chunks. Each score stands for four frames of the original, so its label is copied to all four. Then every active frame is padded by 0.3 seconds on each side, 9 frames at 30 frames a second, so a cut doesn’t clip the start or end of something happening.

The earlier version marked a window around each active frame with a nested loop. The current one sweeps the frames twice, forwards and then backwards, carrying a countdown that restarts at every active frame:

```python
streak = 0
for f in range(result.total_frames):
    if labels[f]:
        streak = margin_frames + 1
    if streak > 0:
        padded[f] = True
        streak -= 1

```

The backward pass is the same loop run from the end. Between them, every frame within the margin of an active frame is marked, and each frame is visited twice whatever the margin. A test classifies 54,000 frames, 30 minutes at 30 frames a second, and fails if that takes a second or more. Finally, runs of frames with the same label become chunks.

On the test clip, the active chunk ran from 14.77 to 20.43 seconds: the scroll with its padding. Everything else was cut, and the real `process` command, run with `--no-dedup` (more on that below), turned 30 seconds into 5.67.

## Fitting a target length

`build_speed_map()` in `src/reelify/speed_map.py` gives idle chunks a speed of 0, which means they’re dropped, and active chunks a speed of 1. If the result is still longer than the target, every kept segment gets the same faster speed:

```python
if total_output > max_duration_secs:
    total_active_secs = sum(
        segment_duration_secs(seg, result.fps) for seg in segments if seg.speed > 0.0
    )
    required_speed = total_active_secs / max_duration_secs
    scaled_speed = min(required_speed, MAX_SPEED)

```

`MAX_SPEED` is 8, so a target can be missed. One of the tests gives it 100 seconds of activity and a 5-second target, and expects 8 times speed, which comes out at 12.5 seconds. On the test clip, `--max-duration 2` asks for 5.67 seconds to fit in 2, a speed of about 2.83. Without `--max-duration`, the command uses a target of 86,400 seconds, a whole day, so nothing is sped up and only the cuts apply. That’s how it’s meant to work: the speed only changes to fit a target, and there’s no separate speed for quiet stretches yet.

## Cutting out the segments

The encoder cuts each kept segment out with FFmpeg, several at once, and joins the pieces with FFmpeg’s concat demuxer. The start and length of each segment go before the input:

```python
half_frame = 0.5 / fps
start_time = max(0.0, start_frame / fps - half_frame)
duration = (end_frame - start_frame) / fps - half_frame

cmd = [
    "ffmpeg",
    "-y",
    "-ss", f"{start_time:.6f}",
    "-t", f"{duration:.6f}",
    "-i", str(input_path),
]

```

Before the input, `-ss` and `-t` are measured on the recording’s own timeline. FFmpeg decodes from the keyframe before the start and throws frames away until it gets there, and because every segment is re-encoded anyway, the cut lands on the exact frame. Both ends sit halfway between two frames, so a timestamp that rounds the wrong way can’t add a frame or lose one. Then the video filter retimes the segment:

```python
vf = (
    f"setpts=(PTS-STARTPTS)/{speed},"
    f"fps={fps}:round=up,"
    "scale=trunc(iw/2)*2:trunc(ih/2)*2"
)

```

`setpts` restarts the segment’s clock at zero and divides it by the speed. `fps` keeps the recording’s frame rate, so at 3 times speed the segment keeps its first frame and every third one after it. The scale rounds an odd width or height down to an even number, which libx264 needs. Sound is retimed with a chain of `atempo` filters of no more than 2 times each, so 8 times becomes `atempo=2.0,atempo=2.0,atempo=2.0000`.

A change in April moved `-ss` after the input for an accurate seek, but there it’s an output option, measured after `setpts` has divided the timestamps, so sped-up segments came from the wrong part of the video or came out empty until I moved both options back. On the test clip with `--no-dedup`, `--max-duration 2` now gives 2 seconds of video, roughly every third frame from 14.77 to 20.43 seconds. The encoder’s older speed test encodes a whole 3-second clip at 2 times and checks that the file is bigger than 1KB, which can’t show the mistake. The new tests encode a clip whose frames show their own number as eight black or white stripes, one per bit, and check exactly which frames come out: frames 20 to 39 at 2 times must give 20, 22, 24 and so on up to 38.

## The dedup step

One step runs before all of this. By default, `reelify process` drops repeated frames first, and the rest of the pipeline works on the shorter video that leaves. It uses FFmpeg’s `mpdecimate` filter, which compares each frame with the last one it kept, in blocks of 8 by 8 pixels, and drops the frame if no block has changed by more than a threshold:

```python
hi = round(64 * 255 * (1.0 - similarity))
lo = round(hi * 5 / 12)
decimate = f"mpdecimate=hi={hi}:lo={lo}"

```

`hi` is a sum of differences over the 64 pixels in a block, so with the default `--dedup-similarity` of 0.90, a block counts as changed when its pixels have moved more than a tenth of the way between black and white, on average. `lo` is a lower threshold that a frame can also cross with enough blocks at once. A moving pointer or a typed character clears `hi` easily, and the encoder’s noise in a still picture stays well under it. On the test clip, the step kept 302 of the 900 frames: the first, the 300 where the pointer moves or the page scrolls, and the first still frame after the scroll. The default command then turned 30 seconds into 5.3, a little less than with `--no-dedup`, because no still frames are left after the scroll to pad it with.

The first version of this step used FFmpeg’s scene score instead, which measures how likely a frame is to start a new scene rather than how much it changed, so it stayed near zero through the scroll, kept a single frame of my clip and stopped the run with “No active segments to encode”.

## Using it

You need Python 3.10 or later and FFmpeg.

```bash
pip install -e .
reelify process recording.webm
reelify process recording.webm --max-duration 60

```

Each writes `recording_summary.mp4` next to the input, and the second speeds it up, if it needs to, to fit in a minute, as long as eight times is fast enough. `reelify analyse recording.webm` writes the segments, scores and keyframe captions as JSON instead, with the captions from a vision model running locally in LM Studio. The code is on [GitHub](https://github.com/tomgould/Reelify).

- **Projects:** [Reelify](https://www.dealmobile.co.uk/projects/reelify.md)
- **Tags:** [Python](https://www.dealmobile.co.uk/writing/tag/python.md), [Testing](https://www.dealmobile.co.uk/writing/tag/testing.md), [Reelify](https://www.dealmobile.co.uk/writing/tag/reelify.md)

---

Index of this site: [llms.txt](https://www.dealmobile.co.uk/llms.txt)