STM32 iPod iPhone Accessories Library - Presentation v0.2

STM32 – iPod/iPhone Accessories Library General Presentation 15th

November Draft 0.2

iPod application - Examples

2

How does it work? – 1/2
Licensing plan:
Customer must be Apple MFi licensee* to get:






“Authentication Coprocessor” (ACP) mandatory 30-pin male connectors Testing/certification tools and softwares A customer notice must be signed and returned to division The customer must also be registered on the audio SharePoint by local ST Sales and marketing people.
After completion of these last 2 steps, the STM32 iPod/iPhone Accessories software can be delivered to the customer. More details will be provided shortly by the division.


STMicroelectronics provides Apple firmware protocol “iPod Accessory Protocol” (iAP) *MFi: “Made For iPod” program – click here 3

How does it work? – 2/2
iPod 30-pin connector can provide audio data in different formats:
Digital audio:
USB host core
USB Audio class for PCM audio data
USB HID class for control and information data


Analog audio:
UART communication for control and information
Direct output of analog audio signals on the 30-pin connector

4

iPod application features iPod Accessory Protocol provided by ST:






Full compatibility * through iPod/iPhone/iPad families IDPS** authentication 2.0B Digital audio (USB 2.0 FS) Analog audio (UART) Automatic switch from USB to UART communication to support older iPod

* : Fully compatibility : the API/library is based on Apple_Release_2010-9. Full Testing, characterization is on-going **IDPS: “Identify Device Preferences and Settings” mandatory for any new iPod accessory

5

Firmware summary Cortex™-M3 CPU 71.88 MHz

64 Kbyte SRAM

DMA

256 Kbyte Flash Volume/ Ch Mixer

iAP Stack Loudness Control 5 bands - Equalizer

Audio Class* HID Class*

HMI Control & Display

I²S PLL block

Audio DAC

Amp

Speakers / Headset

FS USB Host *

USB

UART

SPI

I2C

XTAL oscillators 32 kHz + 3-25 MHz

14.7456 MHz quartz

iPod/iPhone 30-pin connector

Authentication Coprocessor

Touch Screen QVGA LCD

XXX

FW/API from ST

XXX *

Licensed FW from HCC embedded

iPod demoboard overview

7

Connectivity STM32F105 and iPod USB Host Connector

STM32F105 Connectivity Line

Amplifier

USB Host

iPod/iPhone 30-pin connector

USART

I2C I2S

Authentication Coprocessor

Suitable STM32F105 products: -STM32F105RCT6: 256K/64pins -STM32F105RBT6: 128K/64pins -STM32F105VCT6: 256K/100pins -STM32F105VBT6: 128K/100pins

I2C

Audio stream

DAC

Control data 8

iPod “docking” demoboard 6 Key Features:

4

1. Extension connector: straightforward connection to STM3210C eval board via CN1 and CN2 2. iPod/iPhone Authentication Coprocessor 2.0B class 6 (ACP) 3. iPod/IPhone connector 4. Hardware jumpers setting •I2C address selection for ACP module •Accessory Identify : UART or USB •Power select: iPod/iPhone board power source from iPod or STM3210C eval board

5. Video extension connector

3

6. Analog Audio

5

2

•Input analog: Microphone •Audio Pre-amplifier module •Jack connector (analog audio output from iPod)

1

9

STM32 iPod/iPhone control Ref Design (1/2) *MP3 Encoder *MP3 Mixer Media Browser

Media Player WAV, MP3, *WMA

Cursor (Multi-Input) Joystick & Touch 100

Audio Player

60

Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental Year : 1981

Tags Display MP3

Audio Utilities Loudness 5 bands Equalizer

80

Volume Control Using Software Mixer

40 20

Progressive Progress Bar

0:25

0:30

0 Mute

R

*Karaoke Game

iPod Control Info & Credits

Play Pause

Stop

Browser Fast Forward/Backward

Mute Control

Home Repeat

Cursor (Multi-Input) Joystick & Touch 100

iPod Player

Song Info

80 60

Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental

Volume Control Using Software Mixer

40 20

Progressive Progress Bar

0:25

0:30

0

R Play

Mute Mute Control Home

Stop Next/Previous Fast Pause Track Repeat Forward/Backward

Audio Utilities 5 Bands Equalizer

Run-time Utilities While playing

5dB

5dB

5dB

5dB

Loudness Control Max Gain 5dB

2.5dB

2.5dB

2.5dB

2.5dB

2.5dB

0dB

0dB

0dB

0dB

0dB

2.5dB

2.5dB

2.5dB

2.5dB

2.5dB

5dB

5dB

5dB

5dB

5dB

1KHz

3KHz

6KHz

12KHz

18KHz

0

Min Gain Quit

Either Loudness Control Or 4 to 5 bands Equalizers * Unavailable feature

STM32 iPod/iPhone control Ref Design (2/2) Cursor (Multi-Input) Joystick & Touch 100

Audio Player Tags Display MP3

80 60

Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental Year : 1981

Volume Control Using Software Mixer

40 20

Progressive Progress Bar

0:25

0:30

0 Mute

R Play Pause

Stop

Browser Fast Forward/Backward

Mute Control

Home Repeat


iPod control is using digital audio (USB communication)
This demo is NOT using any OS, but based on interrupts priorities
MP3/WAV decoding, EQ, volume control, are all done by software within the STM32

11

iPod firmware mechanism - USB data HCC USB audio class driver

Priority

Audio post processing Transfer complete

SOF period

Mem to I2S transfer DMA transfer DMA transfer DMA transfer DMA transfer DMA transfer USB high priority tasks

IOE

Main loop

Background tasks: (iAP stack and GUI) Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period Asynchronous input events (Joystick & touch screen) 12

iPod demo firmware memory footprint Total Demonstration requirements: Flash: 171.21 Kbytes. Read Only data: 62.82 Kbytes. Code size: 108.39 Kbytes. RAM: 63.68 Kbytes. Heap: 28 Kbytes. Stack: 4Kbytes. Read Write Data: 31.68 Kbytes.

HCC stack resources: Flash: 37.34 Kbytes. Code: 37.16 Kbytes. RO Data: 0.18 Kbytes. RAM : 28.9 Kbytes. GUI: Flash: 19.90 Kbytes. Code: 13.09 Kbytes. RO Data: 6.81 Kbytes. RAM : 5.15 Kbytes. (4 Kbytes heap, 1 Kbytes Stack).

Connectivity Demo: Flash: 73.208 Kbytes. Code: 25.75 Kbytes. RO Data: 47.45 Kbytes. 8 Pictures (48 x 48 16bits). 1 picture (80 x 36 16bits). Zero filled buffer: 4 Kbytes. RAM: 2.03 Kbytes.

MP3 Decoder resources: Flash: 22Kbytes Code: 15 Kbytes. RO Data: 7 Kbytes. RAM (from Heap): 12.45 Kbytes. Equalizer resources: Flash: 1.64 Kbytes Code: 1.52 Kbytes. RO Data: 0.12 Kbytes. RAM : 0.29 Kbytes. Loudness control resources: Flash: 3.16 Kbytes. Code: 1.94 Kbytes. RO Data: 1.22 Kbytes. RAM : 0.61 Kbytes. (scratch : 0.5 Kbytes, persistent 0.11 Kbytes) Mixer resources: Flash: 0.57 Kbytes. Code: 0.57 Kbytes. RO Data: 0 Kbytes. RAM : 16 bytes.

IAR C-standard library: FW HAL Drivers: Flash: 1.55 Kbytes. Flash: 9.75 Kbytes. Code: 1.55 Kbytes. Code: 9.71 Kbytes. RO Data: 2 bytes. RO Data: 38 bytes. RAM: 68 bytes. RAM: 60 bytes

iPod firmware only: Flash: 30Kbytes – RAM: 45Kbytes 13

Back-up Technical details

iPod player(1/8)
iPod Player controls iPod internal player engine
Authenticate with the iPod.
Synchronize screen contents with iPod Player (progress bar, total time, current time, song info).
Sends Play, Pause, Stop, Fast forward, Fast Backward, Next track, Previous track and Repeat commands.
Retrieve PCM data stream from iPod.
Regulate output stream (Volume, audio post processing,...).

15

iPod player(2/8) USB audio streaming Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC(refer to next slides)

Stream synchronization

apply additional filters (Equalizers or loudness control) and volume control

Monitor buffer indexes Yes

Indexes too far/near No

apply additional filters (Equalizers or loudness control) and volume control

Audio data processing 100 x frame length Circular Buffer

Write data to circular buffer Transfer PCM to external DAC via I2S peripheral

Audio stream rendering

iPod player(3/8) :Stream synchronization 

Consumer period < Producer period

Under run condition I2S needs buffer 10 which is not ready

USB transfer

1

I2S transfer

2 1

3 2

4

5

3

4

7

6 6

5

7

8

9

8

9

11

10 10

11

… Accumulative error



Consumer period > Producer period USB transfer I2S transfer

1

2 1

3

2

4

5 3

Over run condition

…

USB prepared buffer 10 while I2S is starting to transfer buffer 9

7

6 4

5

8 6

9 7

10

11

8

9


Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC.

iPod player(4/8) :Stream synchronization -

Method 1 : adjust USB Start of frame (SOF) period to cancel errors and keep USB audio data streaming synchronized with I2S transfer. (default method)

-

This method is enabled by default, to have High end Audio Quality 1:1 ratio ( lossless samples)

iPod player(5/8) :Stream synchronization - Method 2 : a software method to synchronize I2S and isochronous transfer. When needed add one sample at the end of the buffer or remove one sample from the end of the buffer to tune I2S transfer period. Case of adding samples Buffer with N x samples

/2 /2 Buffer with N+1 x samples + ++ + Left Right

New sample (left and right) to be added at the end of buffer

iPod player(6/8) :Stream synchronization Case of removing samples

N-5

Right

N-6

Left

N-1

/4

N-2

N-4

/4

N-3

N-5

N-6

W

N-7

W

N-8

One Sample (left and right) to be removed at the end of buffer

iPod player(7/8) HCC USB audio class driver Audio post processing Priority

Transfer complete

SOF period

Mem to I2S transfer DMA transfer DMA transfer DMA transfer DMA transfer DMA transfer USB high priority tasks IOE Main loop Background tasks: (iAP stack and GUI) Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period

Asynchronous input events (Joystick & touch screen) 21

iPod player(8/8) -

USB tasks scheduling avoid “mutex dead locks” and “concurrency on the USB bus”. Priority USB High priority tasks call HCC Audio task & HCC transfer task

USB high priority tasks USB data out

Time HID report are sent only after higher priority tasks execution

Preemption time Blocked time Running time

Known limitation(1/5)
iPod tracks Tags: Only the Title, Artist, Album and Genre tags are extracted.
For Demo purposes.


iPod control application supports only iPod Accessory Protocol small packets.
For Demo purposes.


Audio quality may be not good with on board speaker, it’s due to the on board speaker quality. For a better experience please use a headphone.

23

Known limitation(2/5)
iPod control track information (stream length, Title, Artiste, Album and Genre) are retrieved when iPod sends new chapter or new track event notification.
iPod control streaming : supported sampling rate are 48KHz, 44.1 KHz and 32KHz as in Apple specifications.
Next/Previous track buttons allow navigation within the selected data base records. 24

Known limitation(3/5)
When MP3 player demo detect Bit rate variable (VBR) it will reset:
Time progress to 0:00
Stream Time length to 0:00
Then "VBR Detected." Message will be displayed.


Media browser: If the mass storage device contains a big number of files some of them may not be displayed.
This is related to the heap space allowed for GUI items and files/folders list. items and list are created dynamically (dynamic memory allocation).

25

Known limitation(4/5)
Maximum FIFO size set to 512 bytes: a received packet larger than 512 bytes will be improperly handled.
Supported lingoes and minimal versions Lingoes General Lingo

Minimal lingo version 1.02

Simple remote lingo

1.00

Display remote lingo

1.02

Digital audio lingo

1.02[1]

1. iPod is not switched to extended mode (please refers to iPod Accessory protocol specification for more info)

26

Known limitation(5/5)
Firmware design switch iPod to USB configuration 2 (composite USB device):
iPod mass storage feature is unavailable when connected to the STM32 docking station.


I2S and USB clock synchronization:
Synchronization method which adjust USB start of frame (SOF) packet period produce better audio quality, however this regulation method produce time lag in SOF period from standard 1ms period > 0.05% due to our I2S drift at 44.1KHz

27