Apple Music Understanding

Spec sheet

Purpose

Music Understanding is an Apple framework for analyzing audio and extracting musical information.Read 14 Nov 2026 · developer.apple.com

Analysis dimensions

It analyzes key, rhythm, structure, pace, instrument activity, and loudness.Read 14 Nov 2026 · developer.apple.com

Export

Session results can be encoded as JSON.Read 14 Nov 2026 · developer.apple.com

File input limit

Asset-based sessions require locally available media or a complete file.Read 14 Nov 2026 · developer.apple.com

Incremental delivery

The framework offers a streaming API that delivers loudness values every 100 milliseconds of analyzed audio.Read 14 Nov 2026 · developer.apple.com

Instrument activity

Instrument activity results can indicate instrument presence and activity intensity over time.Read 14 Nov 2026 · developer.apple.com

Loudness output

Loudness results include integrated, momentary, short-term, and peak measurements.Read 14 Nov 2026 · developer.apple.com

On-device and offline

The framework runs on-device and works offline; Apple says analyzed audio stays private.Read 14 Nov 2026 · developer.apple.com

Rhythm output

Rhythm results include beat and bar timestamps and an optional global tempo in BPM.Read 14 Nov 2026 · developer.apple.com

Selective analysis

Apps can request only selected analysis types to avoid unnecessary computations.Read 14 Nov 2026 · developer.apple.com

Session input

A session can use an AVAsset or a custom audio provider.Read 14 Nov 2026 · developer.apple.com

Streaming input

A custom audio provider yields AVReadOnlyAudioPCMBuffer objects through an AsyncSequence.Read 14 Nov 2026 · developer.apple.com

Structure output

Structure analysis returns time ranges for sections, segments, and phrases.Read 14 Nov 2026 · developer.apple.com

Use cases

Apple suggests beat-synced visuals, DJ catalog organization, and music-driven game animation as use cases.Read 14 Nov 2026 · developer.apple.com

Where Specifiction places it

• AI Song Analyzers#9 of 277.2