is it possible to query native input format

[SchmadenSchmuki]

Hello

When creating a low latency input stream, by looking at aaudio echo, I found out that I had to set the format to oboe::AudioFormat::I16.
By default oboe selected float (which works fine for output) and created a non low latency stream.
(oboe::PerformanceMode::LowLatency was requested, tried it on a Nexus 5X with Android 8.1)
So I was wondering if there is a way to query the needed input format for a low latency inputstream.
Also was wondering if it depends on the android version, the android device or is it always oboe::AudioFormat::I16 for input streams?

two offtopic minor questions I'd like to ask:

• While looking for an answer on the above question I read that aaudio is problematic on some android devices with android 8.0. Will oboe select OpenSL ES for android 8.0 by default or should I do that in my code to be on the save side?
• In the callback, what is the effect of returning oboe::DataCallbackResult::Stop?
  The results I got were as expected on the audio device, for example no more audio output. Still, in the logs I could see that the callback still got called repeatedly. So I was wondering if that's how it should be. From the name oboe::DataCallbackResult::Stop I would have expected the stream to stop (ideally still playing the frames provided in the callbackcall that returned oboe::DataCallbackResult::Stop).

thanks in advance
schmaden

[SchmadenSchmuki]

Hello
Had another problem with low latency inputstreams.
When I play the recorded data from the low latency inputstream it sounds really bad, mangled.
Easily possible that's because of a stupid mistake in the oboe::AudioStreamCallback, but I can't find it. It works with the non low latency inputstream, where the audioformat is float, the callback doesn't much more than casting and copying, essentially it looks like that:

if(oboeStream->getFormat() == oboe::AudioFormat::Float)
{    
    float* sourceAddress(reinterpret_cast<float*>(audioData));
    float* targetAddress(vectorAudioData.data());

    for(uint64_t i(0); i < samplesToRecord;)
    {
        targetAddress[numberOfRecordedSamples++] = static_cast<float>(sourceAddress[i++]);
    }
}
else
{
    int16_t* sourceAddress(reinterpret_cast<int16_t*>(audioData));
    float* targetAddress(vectorAudioData.data());

    for(uint64_t i(0); i < samplesToRecord;)
    {
        targetAddress[numberOfRecordedSamples++] = static_cast<float>(sourceAddress[i++]);
    }
}

vectorAudioData is of type std::vector<float>,
numberOfRecordedSamples is an uint64_t outside the callback that counts the recorded samples of all callbackcalls and
samplesToRecord just checks if there is enough space in the vector to write all the samples provided, so it's usually numFrames * channelcount.
I made the two possibilities as similiar as I could, so now the only difference is how audioData is cast.
The samplerate and channelcount are the same in both cases, so a mistake in handling those should not have been a problem.
The streams differ in audioformat, bytes per sample, bytes per frame, burst size, buffer size and performance mode. The different audioformat is addressed, the rest shouldn't bother me (probably).

So I was wondering, if there is working low latency code with oboe, so I would know that I have to find the solution somewhere in my code (if I can't find the mistake, non low latency inputstream will also do).

[philburk]

Please file separate Issues for different topics. Otherwise it is very hard to track and we cannot close one without closing them all. Please file a separate Issue for "oboe::DataCallbackResult::Stop not stopping the callback". Please include OS version and other details.

[philburk]

I was wondering if there is a way to query the needed input format for a low latency input stream.

If you do not specify the format then Oboe and AAudio should choose the best format. Then you can query it after the stream is opened. There is not currently a way to query before opening the stream.

Unfortunately in AudioFlinger, a FLOAT input stream will not get a FAST path. This is being tracked internally as b/38268031

In P we added code to AAudio to open an Int16 stream and then convert to FLOAT internally.
I added an Issue to convert data formats in Oboe: #276

[philburk]

Please also file a separate Issue for "When I play the recorded data from the low latency inputstream it sounds really bad, mangled." Please include the device and OS. This may be a timing issue for MMAP.

[philburk]

Will oboe select OpenSL ES for android 8.0 by default
Yes. See: #40
And: https://github.com/google/oboe/blob/master/docs/AndroidAudioHistory.md