Hi,

I'm pleased to finally release the algorithm that has been at the heart of the asPitchDetect Xtra for so long, and to release it as open source (under the very open MIT licence).

For now, it comes with an iPhone project, but it is soon to be followed by other projects.

Enjoy !

Hello,

I am trying to use your algorithm for realtime voice pitch tracking. I will send the code in a separate email but for now I have a simple question. In your call to main tracking function, we need to send in 4 parameters. One pointer to the tracker, a pointer to the buffer, THE INDEX OF THE FIRST SAMPLE, and the Number of samples per buffer right?. Somehow I am unable to understand what I should send in the third parameter. Since I am sending a single buffer eachtime the buffer is filled, I am inclined to think that this index should always be 0, (the first of the buffer I am sending) Am I wrong? Somehow when I try that, the algorithm always comes back with a pitch of 0, from nomins and nomaxes part of the routine...

I know this will make more sende once I send you the code but I just wanted to ask what the third and the fourth parameters should be. Thanks in advance for your answer. I will send you the code as soon as I can so it makes more sense. Thanks

hello,
if you always send the same buffer filled with new data, you should indeed give 0 as the third parameter, and the size of the buffer as the fourth parameter (assuming that the buffer has exactely a size equal to the number of samples needed). The function uses these numbers to walk the samples.
A return value of 0 means that it could not find a pitch. You should check that the audio values passed to the function indeed represens sound that has a pitch.
Let me know if you need more help.

Hi again

Here is my audio setup procedure that sets up the sample rate etc

- (void)setupAudioFormatAudioStreamBasicDescription*)format
{
format->mSampleRate = 44100.0;
format->mFormatID = kAudioFormatLinearPCM;
format->mFramesPerPacket = 1;
format->mChannelsPerFrame = 1;
format->mBytesPerFrame =
format->mBytesPerPacket = format->mChannelsPerFrame * sizeof(short int);
format->mBitsPerChannel = 16;
format->mReserved = 0;
format->mFormatFlags = kLinearPCMFormatFlagIsBigEndian |
kLinearPCMFormatFlagIsSignedInteger |
kLinearPCMFormatFlagIsPacked;
}

AND HERE IS MY START RECORDING FUNCTION

- (void)startRecording
{
[self setupAudioFormat:&recordState.dataFormat];

recordState.currentPacket = 0;

OSStatus status;
status = AudioQueueNewInput(&recordState.dataFormat,
AudioInputCallback,
&recordState,
CFRunLoopGetCurrent(),
kCFRunLoopCommonModes,
0,
&recordState.queue);

if(status == 0)
{
for(int i = 0; i < NUM_BUFFERS; i++)
{
AudioQueueAllocateBuffer(recordState.queue,
2940, &recordState.buffers[i]);
//every 30th of a second 44.100 * 2 bytes / 30 = 88200 / 30
AudioQueueEnqueueBuffer(recordState.queue,
recordState.buffers[i], 0, NULL);
}

status = AudioFileCreateWithURL(fileURL,
kAudioFileAIFFType,
&recordState.dataFormat,
kAudioFileFlags_EraseFile,
&recordState.audioFile);
if(status == 0)
{
recordState.recording = true;
status = AudioQueueStart(recordState.queue, NULL);
if(status == 0)
{
labelStatus.text = @"Recording";
}
}
}

if(status != 0)
{
[self stopRecording];
labelStatus.text = @"Record Failed";
}
}



Based on this structure here is my callback input function

void AudioInputCallback(
void *inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp *inStartTime,
UInt32 inNumberPacketDescriptions,
const AudioStreamPacketDescription *inPacketDescs)
{
double thepitch = 0;
double *BufferPointer;

RecordState* recordState = (RecordState*)inUserData;
if(!recordState->recording)
{
printf("Not recording, returning\n";
}

if(inNumberPacketDescriptions == 0 && recordState->dataFormat.mBytesPerPacket != 0)
{
inNumberPacketDescriptions = inBuffer->mAudioDataByteSize / recordState->dataFormat.mBytesPerPacket;
}

BufferPointer = (double*)(&inBuffer);
// is this right? or is it (double*)(inBuffer)
thepitch = dywapitch_computepitch(&mytrack,BufferPointer,0 ,2048);
// the pitch always comes back as 0
//AudioStreamBasicDescription dataFormat;
//AudioQueueRef queue;
//AudioQueueBufferRef buffers[NUM_BUFFERS];
//AudioFileID audioFile;
//SInt64 currentPacket;
//bool recording;

NSLog([NSString stringWithFormat:@"thePtch %d",thepitch]);
printf("Writing buffer %lld\n", recordState->currentPacket);
OSStatus status = AudioFileWritePackets(recordState->audioFile,
false,
inBuffer->mAudioDataByteSize,
inPacketDescs,
recordState->currentPacket,
&inNumberPacketDescriptions,
inBuffer->mAudioData);
if(status == 0)
{
recordState->currentPacket += inNumberPacketDescriptions;
}

AudioQueueEnqueueBuffer(recordState->queue, inBuffer, 0, NULL);
}

WHAT AM I DOING WRONG PLEASE HELP

Hello,
your allocated buffers are shorts (16 bits), and the dywapitch_computepitch function requires doubles. You need to convert your shorts into doubles before passing them to dywapitch_computepitch. For this you can use a temporary buffer for example, or allocate one once and for all and reuse it all the time.
You can look into the iPhone example that is in the delivery : SpeakHere/Classes/AQRecorder.mm for an example.

Also I advise that you allocate bufers that have a size that is a power of two (not 2940). It usually works better for all sound routines. This is just an advice, not a requirement. This way also, you will pass all the sound bytes to the dywapitch_computepitch function, which will probably lead to better results.

Cheers !

Hello once again Antoine. I know I may be asking a little too much but I am new to the language and I am a little lost here.

Fior your suggestion, I have changed the buffer allocations to teh following


for(int i = 0; i < NUM_BUFFERS; i++)
{
AudioQueueAllocateBuffer(recordState.queue,
4096, &recordState.buffers[i]);
AudioQueueEnqueueBuffer(recordState.queue,
recordState.buffers[i], 0, NULL);
}

in order to accomodate a power of 2 there fore sending

thepitch = dywapitch_computepitch(&mytrack,BufferPointer,0, 4096);

as the function call. Now bare with me here, since the function requires doubles and my samples are shorts

before calling the function do I declare a temp buffer and each time the callback is called pass the data to that doubles buffer. Although I understand the concept my knowledge of obj is not satisfactory. Will you let me know how.

Essentially I am understanding that in the routine


void AudioInputCallback(
void *inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp *inStartTime,
UInt32 inNumberPacketDescriptions,
const AudioStreamPacketDescription *inPacketDescs)
{
double thepitch = 0;
double *BufferPointer;

RecordState* recordState = (RecordState*)inUserData;
if(!recordState->recording)
{
printf("Not recording, returning\n";
}

if(inNumberPacketDescriptions == 0 && recordState->dataFormat.mBytesPerPacket != 0)
{
inNumberPacketDescriptions = inBuffer->mAudioDataByteSize / recordState->dataFormat.mBytesPerPacket;
}

BufferPointer = (double*)(&inBuffer);
thepitch = dywapitch_computepitch(&mytrack,BufferPointer,0, 4096);

//AudioStreamBasicDescription dataFormat;
//AudioQueueRef queue;
//AudioQueueBufferRef buffers[NUM_BUFFERS];
//AudioFileID audioFile;
//SInt64 currentPacket;
//bool recording;

NSLog([NSString stringWithFormat:@"thePtch %d",thepitch]);
printf("Writing buffer %lld\n", recordState->currentPacket);
OSStatus status = AudioFileWritePackets(recordState->audioFile,
false,
inBuffer->mAudioDataByteSize,
inPacketDescs,
recordState->currentPacket,
&inNumberPacketDescriptions,
inBuffer->mAudioData);
if(status == 0)
{
recordState->currentPacket += inNumberPacketDescriptions;
}

AudioQueueEnqueueBuffer(recordState->queue, inBuffer, 0, NULL);
}


instead of the line that says

BufferPointer = (double*)(&inBuffer);

I need to pass this data to a doubles buffer declaring a temp buffer ?

Please tell me how, and I will not bother you again I promise... How do I declare that temp buffer and pass the data to that double

Hi
I will not be in front of my computer for the coming days and It is difficult to type code on my iPhone.

You can look into the iPhone example that is in the delivery : SpeakHere/Classes/AQRecorder.mm for an example.

The key point is that in C, casting does convert a short into a double, but not an array of shorts into an array of doubles. You need to loop over the arrays and convert individual values:

for (int i = 0; i < nbsamples ; i++)
mydoublearray[i] = (double)myshortarray[i];

You need to allocate your double array in the init function. Using malloc() for example.

Good luck !

Thank you so much Antoine. I will try this tomorrow... Although instead of doing this would it not be easier to configure the arrays as doubles in the beginning instead of shorrts, not that I know how to do that either As I said I am pretty new to the Objective C. Please bare with me if I bother you too much... You have been so kind...

Hello again
Here is my description

format->mSampleRate = 44100.0;
format->mFormatID = kAudioFormatLinearPCM;
format->mFramesPerPacket = 1;
format->mChannelsPerFrame = 1;
format->mBytesPerFrame =
format->mBytesPerPacket = 2 ;//format->mChannelsPerFrame * sizeof(double);
format->mBitsPerChannel = 16;
format->mReserved = 0;
format->mFormatFlags = kLinearPCMFormatFlagIsBigEndian |
kLinearPCMFormatFlagIsSignedInteger |
kLinearPCMFormatFlagIsPacked;

ASIDE FROM THAT I AM ALLOCATING MY BUFFERS AS THE FOLLOWING


AudioQueueAllocateBuffer(recordState.queue,
8192, &recordState.buffers[i]);


SO THIS SHOULD GIVE ME 4096 SAMPLES FOR EACH BUFFER

NOW TO CONVERT THE SAMPLES TO THE DOUBLE THAT YOUR ROUTINE REQUIRES, I HAVE DECLARED

double *MyDoubleArray;

AND INITIALIZED IT IN MY INIT AS SUCH


- init {
if (self == [super init]) {
MyDoubleArray = malloc(4096 * sizeof(double));
// Your initialization code here
}
return self;
}

SO FROM HERE ON IT IS JUST THE FOR LOOP THAT I NEED TO WRITE IN ORDER TO CONVERT THE ARRAY.


for (int i =0;i<4096;i++){
MyDoubleArray[i]= (double)????????/
}

NOW I KNOW THERE SHOULD BE AN ARRAY WHERE THE ???? ARE BUT I AM UNABLE TO FIND AN ARRAY STRUCTURE ANYWHERE RELATED TO TEH CURRENT BUFFER DATA

HERE IS THE CALLBACK FUNCTIONS DEFINITION LINE

void AudioInputCallback(
void *inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp *inStartTime,
UInt32 inNumberPacketDescriptions,
const AudioStreamPacketDescription *inPacketDescs)
{

THE INUSER DATA IS A VOID * IS THAT WHERE MY SAMPLES ARE ))

I AM SO SORRY BUT NOT BEING FAMILIAR WITH THE LANGUAGE BRINGS SUCH PROBLEMS. I CHECKED THE EXAMPLE YOU SUGGESTED IT DOES PRETTY MUCH THE SAME THING SO DOES NOT GIVE WHAT SHOULD BE CONVERTED TO DOUBLE...

YOU HAD SAID THAT MY SAMPLES ARE SHORTS, WHERE DID YOU SEE THAT DESC. I AM THINKING THAT IS WHERE MY ARRAY TOBE CONVERTED SHOULD BE.. PLEASE HELP WHEN YOU HAVE THE TIME

This is a proper way to do it :

// this creates an array of short pointer that pointer to the audio data of the incoming buffer. It forces C to consider mAudioData as an array of shorts.
short *audioDataAsShort = (short *)inBuffer->mAudioData;

// then you can convert to double :
for (int i =0;i<4096;i++){
MyDoubleArray[i]= (double)audioDataAsShort[i];
}

That should do it.
This is very basic C. I advise that you learn about these conversions systems on google !

Thank you so much Antoine

I am actually quite familiar with C. I just could not figure out where the data to be converted was going to be. This answers my question I will try this and let you know of the glorious result Thanks again

Hello Antoine... With your help I was able to get a responding double number from your wonderful Pitch tracker. But there is small problem :

Assuming the settings I mentioned above, with a sample rate of 44100 sample rate / second; a double array of 4096 doubles, a buffer size of 8192 bytes (thus 2 bytes for each sample) and the call to pitch tracker as

thepitch = dywapitch_computepitch(&mytrack,Mydoublearray,0,4096)

I am receiving returning pitches in the range 1500-1600 usually. From time to time it will return pitches in the range of 400 or so but usually responding to my voice, the returned pitch is around the 1500-1900 range.

Now, if this is in Hz, the pitch does not respond to the range that voice is giving the mic. Clearly I am doing something wrong. Do I have to do anything else after receiving the pitch in order to use it and make it reasonable or am I messing up my arrays with the wrong buffer size etc. Essentially what I am trying to do is, evaluating a pitch from users voice this and representing it as a mucical note based on the returned pitch like A. B flat, C etc. Any ideas as to what I may be doing wrong will be appreciated immensely. Thank you so much for your incredible patience with me...

The only thing that looks fishy in your code is the kLinearPCMFormatFlagIsPacked flag. A packed short considered as a short will not convert properly to a double. I would try removing this flag. If this does not work, I would look into the Endian flag and be sure that the endianness is the same as the one of the platform.
Good luck !

Hello Antoine... I have tried omitting the packed property but no luck. This app is running on the iphone simulator which I read on the net as being little endian. Is that true? Supposing it is I omitted the bigendian parameter as well that screwed up everything, causing the app to not even fill the buffers appropriately. I feel we are v ery close to the end so please bare with me.

By the way I may be totally out of line here but. I am writing the app on the teh iphone mcbook air using the iphone simulator for the iphone. I do not know how that affects the endianness of the platform. Thanks

Hi,
well, I don't know really.

As I said twice already : please look into the AQRecorder.mm example that is delivered with the dywapitchtrack library. It is written for iPhone and it works. Look into the mFormatFlags that are used, and the other format values, etc... It should work.

Also : are you sure that you call dywapitch_inittracking ?

Hi Antoine. I am calling the init tracking function when the user presses the record button so I don't think that is the problem. As far as the AQRecorder project goes, which u say is delivered by the dywapitchtrack, I may be missing something here corrrect me if I am wrong, but when I download the dywapitchtrack.zip from you, file I get a folder "iphone" in which there is a "dywapitchtrck.xcodeproj" folder and in that folder there is "project.pbxproj" file. I dont think that is the one I am looking for right? Or should I download the AQRecorder.mm from some other place and integrate your code into that project... Thanks

Hi,
yes, sorry, I thought that I had included this testbed in the dywapitchtrack delivery, which is not the case. I am sending you this testbed by email right now.

Thanks Antoine. I will try this once I get home to my Air. After I open xcode I need to open the project.pbxproj in the "SpeakHere.xcodeproj" library right?

Actually, you just open SpeakHere.xcodeproj (which is a package) using xcode. And look into AQRecorder.mm . The SpeakHere project is a standard demo from Apple, which I tweaked a little to do the pitchtracking, so it does a lot more than pitchtracking, just discard the rest...

ok this seems to be working. Although from time to time when I am quiet I suppose it picks up some sounds and computes a pitch I will look at the code try to understand it, and ask you if I need anymore help which I hope I do not. Thank you so much for all the help you have given so far. Cheers !

I put the string '[AS]' in comments where I added some code to the original demo.

Good Luck !

Hi Antoine... All seems to be OK. You have been wonderful. Thanks a million

Hello Antoine... Is there a spesific reason why the pitchtrack will not recognize freq over a certain number like 1300s or so? And if so How can i Make it so it recognizes freq upto say 3000s or so thanks. It seems that the function does not retun any pitch over 1400s or so

Interesting. Normally the algorithm is designed to track pitchs up to 3000Hz. But it is true that I've never tested it above 1000Hz (normal human voice).
WHat are you testing it with ? What kind of sound ?

I am using an electronic piano software downloaded from the app store while testing the software. I did see numbers such as 1200s or 1300s but nothing higher

Maybe this instrument has specific harmonics that make it difficult to trak when above a certain frequency ? Try with a simpler instrument. Also, try to make it louder when in high pitches, volume is important to track pitch...

I will do that Antoine. BUt we tried it with a few piano apps always getting the same result. It was not that it did not get a pitch... It does return a number but after a certain octave, regardless of the note you press it returns a pitch the highest around 1300s. Could the code have a bug in that sense?

I dont think that there is abug. Please try with other instruments, especially with some that do not have a string attack like the piano. Also, as I said, it is important thqt there is volume, and pitch notes are not very stron in volume.

OK I will do that and let you know. I will actually try it with a real violin and see what happens and let you know. Thanks