I’m currently optimizing my timestretcher. Just to give an idea of the speed we need to achieve: with a windowsize of 4096 samples and an overlap of 16 frames. We have about 5.8 milliseconds between frame calculations. Now, this is stereo which means that we only have 2.9 milliseconds/frame. In each frame we have to detect sine waves, extract them relocate them to other positions. Typically we have around ~600 peaks per frame. That leaves us about 4.8 microseconds per peak. Each peak must be detected, extracted, repositioned, resampled and added to the final frame again. Currently I manage to do this in 13 microseconds. And now the push is on. Can we speed this further up with a factor 2.7 ?
All posts by werner
Too hot too handle
BpmDj at Chaos Communication Camp 2015
This year we will again be at the Chaos Communication Camp 2015 (which takes place around Berlin)
At CCC 2011, we gave a talk on how BpmDj performs its audio analysis. Two years later, at OHM 2013, we explained how the nearest neighbor detection and associated weight matrix is created. This year, we won’t talk about the project anymore, but instead give you the opportunity to meet the developers.
Actually, we want you to come to us with ~100 tracks and an idea for a mix. We will then sit together and create that mix. As a reward you will receive one of our heat sensitive ‘stay-tuned-stay-sharp-keep-mixing’ cups.
Updated version of the audiosplitter
The old VCC/2 line, which I originally made with a second opamp has been replaced with a 100K resistor pair. Not the best solution because now it hums even louder. The grounds and the opamp bias have been properly decoupled this time (still hum). Some of the capacitor values have been lower in cutoff frequency (still hum). I tried placing a resistor over the input to remove the hum. That works if the resistor is ~680 Ohm. Yet in that case, if there is an input signal, it is barely amplified (no hum, no signal). Putting the entire thing in a metal box did not do a thing. So no pickup of magenetic fields (still hum). Replacing the powersupply with a batterypack resulted in >huray< no hum ! So it is most certainly a power supply problem. Hum removal is a non trivial problem. Grmpf.
Prototyping the Audio-splitter
IAB startSetup NullPointerException
Another fine example of Googles ‘write once, let others fix our problems’. Even at this moment (June 2014) the ‘patch’ is still not in the released sdk. To solve the problem replace
if (!mContext.getPackageManager().queryIntentServices(serviceIntent, 0).isEmpty()) {
with
PackageManager pm=mContext.getPackageManager();
List<ResolveInfo> intentServices = pm.queryIntentServices(serviceIntent, 0);
if (intentServices != null && !intentServices.isEmpty())
Then in the dispose code
replace
if (mContext != null) mContext.unbindService(mServiceConn);
with
if (mContext != null && mService!=null) mContext.unbindService(mServiceConn);
I hope this helps.
The Symbian/Qt/Nokia platform. A total nightmare.
In June/July 2011 I had the misfortune to program for a Symbian/Nokia phone. Really bad experience. Below a list of all the shit I had to go through to get QtCreator working somewhat.
- crappy error messages. ‘a data abort exception has occured’
- qml is a complete paradigm shift
- unstable operating system. ‘it might be in an unstable state if you interrupt the working of this program’
- version control unfriendly. Often a file added by somebody else must be readded in the repository which leads to duplicate entries in the project file.
- memory mapping of large files doesnt work
- memory requirements are too stringent
- debugger doesn’t work. One always gets memorydumps without access to the symbols.
- bloody slow compilation
- the X7 Symbian 3 phone is slow.
- Try to get the audiostream from a decoder. With phonon it should be possible but it doesnt work.
- just click on the ‘documentation’ tab left. You get an error ‘Die Seite konnte nicht gefunden werden ‘qthelp://com.nokia.qtcreator.220/qdoc//doc/index.html
- sometimes code gets hidden and cannot be unhidden unless the project is opened and closed.
- how can one create a subfolder ?
- copying files to and from the phone more often than not does not work.
- try to use phonon or qtmultimedia to access the decoded part of a video !
- searching in all project files simply doesn’t work.
- qt creator often seems to require compilation twice. ‘build’ and then ‘run’, leads to a rebuild.
- no linux, nor vmware. You must program in windows
- QtCreator crashes when there are more than 4000 error messages from the compiler.
- sometimes when adding a new file, it insists to add it somewhere in a subdirectory.
- The arm processor in those phones is not fully functional. For instance the ‘smull’ and ‘adc’ instructions are not supported. Mainly because it is based on a too old model of the processor.
- ‘piss-poor’ headphone volume; something many buyers noted as well
- Filenames (without their paths) are used to identify files. So the same filename at twop different locations is treated by the make system as the same file.
- How to set the volume of th phone ? Yes a truly horendous hack by obtaining the address of a undocumented function and then hoping that it will actually point to the right thing ! http://bugreports.qt.nokia.com/browse/QTMOBILITY-830
- No support from Nokia what_so_ever. Even if you are hired by them (which I inderectly was !)
I was so happy that I could stop programming for/on this utter swamp of a platform.
A cool job description
A cool job description by the one running APOD (Astronomy picture of the day). Copied from http://asterisk.apod.com/viewtopic.php?f=28&t=23066
APOD Graduate Student Research Assistantship
Post by RJN » Thu Mar 03, 2011 4:12 pm
Applicants are sought for a graduate student research assistantship opening in the Department of Physics at Michigan Technological University. The successful applicant will be expected to complete courses and research for a Ph D. in astrophysics while supporting the production of the Astronomy Picture of the Day (APOD) website. Key attributes sought in applicants include a desire to produce original research and to effectively communicate astronomy and astrophysics to the public.
Interested potential applicants should send an initial email to Prof. Robert Nemiroff (nemiroff at mtu dot edu) mentioning their interest and background. Dr. Nemiroff’s research interests include the investigation of gamma-ray bursts, cosmology, sky monitoring, and gravitational lensing. If encouraged to proceed, applicants should complete all of the application requirements found at the Michigan Tech’s Dept. of Physics website found here: http://www.phy.mtu.edu/physicsgradprog.html . Additionally, applicants should submit an original science or science-fiction writing sample. Preference will be given to applicants with research interests in common with Prof. Nemiroff. For background on Prof. Nemiroff’s research interests, please see http://www.phy.mtu.edu/faculty/Nemiroff.html and http://adsabs.harvard.edu/cgi-bin/nph-a … &version=1 . Applications completed before March 30 may also be given preference. – RJN
How to know whether a copy-on-write page is an actual copy ?
When I create a copy-on-write mapping (a MAP_PRIVATE) using mmap, then some pages of this mapping will be copied as soon as I write to specific addresses. At a certain point in my program I would like to figure out which pages have actually been copied. There is a call, called ‘mincore’, but that only reports whether the page is in memory or not, which is not the same as the page being copied or not.
In order to figure out what pages have been copied, the pagemap and kpageflags can be used. Below a quick test to check whether a page is in memory ‘SWAPBACKED’ as it is called. One problem remains of course, which is the problem that kpageflags is only accessible to the root.
int main(int argc, char* argv[])
{
unsigned long long pagesize=getpagesize();
assert(pagesize>0);
int pagecount=4;
int filesize=pagesize*pagecount;
int fd=open("test.dat", O_RDWR);
if (fd<=0)
{
fd=open("test.dat", O_CREAT|O_RDWR,S_IRUSR|S_IWUSR);
printf("Created test.dat testfile\n");
}
assert(fd);
int err=ftruncate(fd,filesize);
assert(!err);
char* M=(char*)mmap(NULL, filesize, PROT_READ|PROT_WRITE, MAP_PRIVATE,fd,0);
assert(M!=(char*)-1);
assert(M);
printf("Successfully create private mapping\n");
The test setup contains 4 pages. page 0 and 2 are dirty
strcpy(M,"I feel so dirty\n");
strcpy(M+pagesize*2,"Christ on crutches\n");
page 3 has been read from.
char t=M[pagesize*3];
page 1 will not be accessed
The pagemap file maps the process its virtual memory to actual pages, which can then be retrieved from the global kpageflags file later on. Read the file /usr/src/linux/Documentation/vm/pagemap.txt
int mapfd=open("/proc/self/pagemap",O_RDONLY);
assert(mapfd>0);
unsigned long long target=((unsigned long)(void*)M)/pagesize;
err=lseek64(mapfd, target*8, SEEK_SET);
assert(err==target*8);
assert(sizeof(long long)==8);
Here we read the page frame numbers for each of our virtual pages
unsigned long long page2pfn[pagecount];
err=read(mapfd,page2pfn,sizeof(long long)*pagecount);
if (err<0)
perror("Reading pagemap");
if(err!=pagecount*8)
printf("Could only read %d bytes\n",err);
Now we are about to read for each virtual frame, the actual pageflags
int pageflags=open("/proc/kpageflags",O_RDONLY);
assert(pageflags>0);
for(int i = 0 ; i < pagecount; i++)
{
unsigned long long v2a=page2pfn[i];
printf("Page: %d, flag %llx\n",i,page2pfn[i]);
if(v2a&0x8000000000000000LL) // Is the virtual page present ?
{
unsigned long long pfn=v2a&0x3fffffffffffffLL;
err=lseek64(pageflags,pfn*8,SEEK_SET);
assert(err==pfn*8);
unsigned long long pf;
err=read(pageflags,&pf,8);
assert(err==8);
printf("pageflags are %llx with SWAPBACKED: %d\n",pf,(pf>>14)&1);
}
}
}
All in all, I’m not particularly happy with this approach since it requires access to a file that we in general can’t access and it is bloody complicated (how about a simple kernel call to retrieve the pageflags ?).
Synchronizing a wiki between 2 machines
Getting a wiki from the server
#!/bin/bash if [ -e wiki_is_local ]; then echo "Wiki is already locally instantiated" exit fi echo "Downloading files" rsync -xavz --exclude LocalSettings.php root@sigtrans.org:/home/nens/wiki /home/nens/ echo "Downloading database" mysqldump -C --host=213.239.213.249 --user=NensWiki --password=passwd NensWiki >server_state.sql echo "Inserting in local database" mysql -h localhost -u NensWiki --password=passwd -D NensWiki
Putting a wiki towards the server
#!/bin/bash echo "Creating local copy of database" mysqldump -C --host=localhost --user=NensWiki --password=passwd NensWiki >local_state.sql echo "Synchronizing mediawiki directory" rsync -xavz --exclude LocalSettings.php /home/nens/wiki root@sigtrans.org:/home/nens/ echo "Synchronizing remote database" mysql -C -h 213.239.213.249 -u NensWiki --password=passwd -D NensWiki




