<chapter id="multichannel">
<title>Handling Multichannel Data</title>
<para>
&AL; is foremost an API to control processing of spatialized sound. For that
reason, the internal, canonical format of data stored in buffers is mono.
The specification does not require the implementation to preserve the
original data, thus multichannel (e.g. stereo) data is usually downsampled
using an algorithm of the implementor's choosing. Implementations are free
to defer decompression, decoding, and conversion until the latest possible
moment, but they are not required to do so by the specification.
</para>
<para>
However, &AL; is an audio API, and consequently has to provide means to
pass through preprocessed multichannel data to the sound driver and
hardware, which in turn will map it to the actual output channels. This
processing might involve mapping from n channels in the data to m channels
in the output, as well as filtering to account for varying requirements
for a given setup. For example, headphone and speaker setups differ with
respect to handling of crosstalk and noise.
</para>
<para>
For this reason, mono buffers can not be used to store data meant for
direct multichannel output. The API has to provide means by which the
application can signal its intention to use a given buffer in this way,
causing the implementation to either preserve the original data, or
convert it as needed for the direct output to the given speaker or
headphone configuration. In many cases, multichannel buffers could be
used with Sources, but the specification does not endorse this at this
time as this might not be guaranteed for all multichannel data formats
&AL; will support.
</para>
<para>
While spatialized sound is often played once, or looping a limited number
of times, multichannel data is usually streaming: typically stereo or MP3
music, soundtracks, maybe voice data accompanying animation and cinematic
sequences. The queueing mechanism defined for mono buffers and spatialized
Sources is thus also applied to multichannel buffers and passthrough
output.
</para>
<para>
Applications can expect &AL; implementations to support more than one
multichannel output at once (e.g. mixing two stereo streams, or fading
from one to the other for transition). For that reason we can not restrict
multichannel output to just one stream or buffer at a time.
</para>
<![ %Annote [
<note><title>Annotation (per-case Buffer use)</title><para>
The specification will likely be amended at a later time to
allow for the use of mono buffers with multichannel output,
and multichannel buffers with spatialized Sources.
The decision to use the same buffer API for mono
and multichannel buffers was taken in anticipation of this.
In cases where the specification can not guarantee
use of a given multichannel data encoding with a
spatialized source, an error can be generated at the
attempted use, validating the buffer's internal format
against the output chosen. In cases were the specification
can not guarantee the implementation will defer the
conversion of multichannel buffer content as needed (e.g.
PCM stereo data), the application can either specify the
internal format, or the internal format enumeration
can be extended to explicitely request storage of both
multichannel and mono data within the same buffer, at memory
expense.
</para></note>
]]>
<section>
<title>Specifying Buffer Content and Internal Format</title>
<para>
The existing BufferData command does not permit the application
to specify an internal format. The implementation is free to
apply conversions, downsampling, upsampling, including
interpolation and filtering, in accordance with the specification.
</para>
<para>
However, applications might want to use different trade-offs
between quality and resource expenditure. More imortantly,
applications might choose to use preprocessed, multichannel
data instead of doing runtime spatialization, e.g. for
recorded music, voice, and realtime streams. In this case
the application expects efficient passthrough of the data
to the hardware.
</para>
<para>
To account for this requirement, an extended command to specify
sample data to be stored in a given buffer is introduced.
<funcsynopsis><funcprototype>
<funcdef> &void; <function> BufferWriteData</function></funcdef>
<paramdef> &uint; <parameter>bufferName</parameter></paramdef>
<paramdef> &enum; <parameter>format</parameter></paramdef>
<paramdef> &void;* <parameter> data </parameter></paramdef>
<paramdef> &sizei; <parameter> size </parameter></paramdef>
<paramdef> &uint; <parameter>frequency</parameter></paramdef>
<paramdef> &enum; <parameter>internalFormat</parameter></paramdef>
</funcprototype></funcsynopsis>
The internal format of the Buffer can be requested by the
application. The implementation is not required to match
the request excatly. The specification does guarantee that
use of a multichannel internalFormat parameter will cause
the implementation to treat the data as multichannel data.
The implementation is free to perform any remapping from
the number of channels used in the data to the number of
channels available at the output, and apply other prcessing
as needed.
</para>
<para>
Valid formats for internalFormat are FORMAT_MONO8, FORMAT_MONO16,
FORMAT_STEREO8, and FORMAT_STEREO16. An implementation may
expose other formats, see the chapter on Extensions for
information on determining if additional formats are supported.
In general, the set of valid internalFormat parameters will be
a subset of those available as format parameters.
</para>
<![ %Annote [
<note><title>Annotation (atomic WriteData)</title><para>
The internal format for a buffer can not be specified
by using Bufferi. The modular way to set state
is preferable for operations that are always, or
nearly legal and orthogonal, but specifying the
internal format affect allocation and use of a buffer,
and would cause too many error cases.
</para></note>
]]>
<![ %Annote [
<note><title>Annotation (change buffer format on write)</title><para>
The API allows for changing a buffer on every write operation,
turning a multichannel buffer into a mono buffer and vice versa.
The specification does not require the implementation to handle
these operations efficiently, as they might require reallocation
of memory. The same is true for resize operations at write.
Applications might prefer to release buffer names that do not
match current requirements, and request new buffer names instead,
an operation which implementations are encouraged to optimize.
</para></note>
]]>
<![ %Annote [
<note><title>Annotation (BufferReadData)</title><para>
The specification might be amended at a later time
with a command to retrieve the data from a buffer,
following use of alGetBuffer to retrieve size and
internalFormat. Reading has to be queued for
continuous reads from the actual mixing buffer.
</para></note>
]]>
<![ %RFC [
<note id="rfc-bk000813-01"><title>RFC: frequency for write/read? </title><para>
Do we (ever) want to provide the application control over the internal frequency?
</para></note>
]]>
</section>
<section>
<title>Rendering Multichannel Buffers</title>
<para>
For a variety of reasons, Sources are not used to feed buffer
data into multichannel processing. Instead, a different class
of &AL; object is introduced to encapsulate state management
and resource management for multichannel output: Multichannel
objects.
</para>
<para>
Multichannel objects are referred to by name, and they
are managed by an API partly duplicating the Source API:
GenMultichannels, IsMultichannel, DeleteMultichannels,
Multichanneli and GetMultichanneli. However, the
SourcePlay/Stop commands as well as the Queue/Unqueue
commands can be applied to both: these commands accept
mcName parameters as well as sName parameters. It is
left to the implementation how a Name is mapped to an
object, and how the implementation dinstiguishes sNames
from mcNames. Within a 32bit namespace, and given the
small number of multichannel sources to expect (more than
one is quite possible, but not common), the implementation
can exploit the fact that Names are opaque to the
application.
</para>
<![ %Annote [
<note><title>Annotation (Source NONE)</title><para>
The literal value "0", i.e. NONE, is legal. All
operations on Source/Multichannel NONE are legal NOPs
and quietly ignored. Because of the ambiguity
(is NONE a Source or Multichannel) NONE will never
be available as an actual object, nor will operations
on it ever have effects.
</para></note>
]]>
<![ %Annote [
<note><title>Annotation (implicit ambient)</title><para>
One way to feed multichannel data into the rendering is
using the Listener as a Source (sname==0 implies
Listener). This was rejected, as it
does not allow for multiple streams. Using multiple
Sources with appropriate type SOURCE_AMBIENT or
SOURCE_MULTICHHANEL was rejected, as Sources
are by definition spatialized, and support operations
which are meaningless or even illegal for an output
that is not spatialized at all. Modifying the semantics
of Source spatialization by a provision that a relative
position of (0,0,0) would mark a Source as ambient was
rejected for clarity and cleanness. Updates of Source
position can not be expected to convert Sources from
spatialized to ambient and vice versa at any time,
a Source State dependent semantics of position updates
is outright confusing, handling of sources that are
SOURCE_ABSOLUTE but incidentally at listener position
can not be different from that of SOURCE_RELATIVE with
a zero offset. Furthermore, multichannel data is not
necessarily ambient (surround can rotate, while ambient
attempts to prevent any perception of direction).
Finally, multichannel processing might require per-channel
gain adjustment at user request, which is a meaningless
concept for Sources.
</para></note>
]]>
<![ %Annote [
<note><title>Annotation (no direct mixing)</title><para>
Direct mixing into the rendering buffer, allowing the
application to superimpose its multichannel data
(or custom software mixing results) on the audio stream
generated by spatialization was rejected, as it suffers
from similar problems as BufferAppendData and loop points,
namely the need to operate at sample level on buffers of
unknown format that might not even be directly accessible.
The same rationale that led to the adoption of maintaining
a processing queue for Sources applies to multichannel data.
</para></note>
]]>
<![ %Annote [
<note><title>Annotation (no combined Buffer/Passthrough)</title><para>
Multichannel buffers cannot serve as multichannel output
objects for two reasons: it would complicate the buffer
API with operations only applicable to a subset of buffers,
and it does not allow for queueing.
</para></note>
]]>
<![ %RFC [
<note id="rfc-bk000813-02"><title>RFC: Per-Channel access</title><para>
At a later time we might want to amend the specification to
gain access to a single channel within a multichannel buffer.
I suggest that any such operation should be restricted to the
ability of reading samples from just one channel into application
memory, which allows for writing it into a mono buffer subsequently.
</para></note>
]]>
</section>
</chapter>
