Python Challenges

For me to learn a programming language, I have to use it. For most people this is likely the case. When I decided to learn Python a couple days ago, I started looking for some fun projects/exercises to do to try and apply my skills. I came across a really cool site that will certainly give even a seasoned Python programmer a run for his money.

It's called Python Challenge. It's a series of puzzles that in order to get the URL to the next puzzle, you must solve the current one. The URL will somehow be the solution to the puzzle. I have to admit that so far they have been rather challenging. They all involve writing a short program to come up with the solution. Most of the solutions are only about 10 - 30 lines of code and don't involve alot of obscure Python tricks, they are more focused on the problem solving aspect.

I encourage anybody looking for a fun challenge to check them out. You really could solve these problems in any language you like, but the solutions to the problems are posted in Python (although people have given alternate solutions in other languages like Shell and Ruby).

Getting Into Python

I have decided that I need a good scripting language in my repertoire. I find that even in my physics courses it's good to have something to do some quick data analysis or to do some numerical calculations. These are obviously perfect uses for a scripting language.

I've decided to go with Python for a couple reasons. The first is that it has alot of support in the scientific community. The two big languages in physics today are C++ and, even after 60 years, FORTRAN. These languages have the power and the library support for scientific computing. However, I am hearing more and more people talk about Python. Some of the grad students here use it for alot of their data analysis.

I would be leery of using it as a substitute for C++ since I can't imagine that even compiled Python is very fast. However, I'm thinking on adopting it as my language of choice for quick "back of the envelope" type computing.

What do you guys think? Has anybody used Python for physics? I'm curious to know how many people use it for heavy duty computing or where you draw the line and just decide to use C++ or some other high powered language.

Converting M4A to MP3

Today I started using Songbird on my Linux box. However, for some reason it doesn't want to play .m4a files. I looked into the solutions out there a bit but I didn't find anyhing that seemed to offer a solution. So my solution was to write a quick script to convert my .m4a files into .mp3 files. For this solution, you will need to have 3 programs installed

• faad2
• lame
• id3cp (comes with id3lib)

The script will take any input files you give it, convert them from m4a to mp3 and then copy all of the ID3 tags over to the new file. It then moves the old m4a files to a new subdirectory and changes all the file extensions to mp3.

mkdir old_m4a_files

for i in $@
do
    echo "Converting: ${i%.m4a}.mp3"
    faad -o - "$i" | lame - "${i%.m4a}.mp3"
    id3cp "$i" "{$i%.m4a}.mp3"
    mv "$i" "old_m4a_files/$i"
done

Thoughts on "Thoughts On Flash"

As some of you may know, Steve Jobs has written an explanation on why Apple does not support Flash on the iPod, iPhone or iPad. You can find the original post here. I just wanted to post some of my thoughts on Jobs' comments.

"Adobe claims that we are a closed system, and that Flash is open, but in fact the opposite is true."

I've see alot of people criticizing Jobs for making this statement claiming that Jobs is being hypocrite (since the iPhone is clearly not an open platform the way that say Linux is). However, if you look into his explanation, he makes his definition quite clear. The iPhone is a closed platform but the languages used for development are completely open. Anybody can go and write a program in Objective-C or JavaScript, all you need is a compiler/interpreter. Flash is a closed language. You can't even write Flash programs unless you use Adobe development software. This means the language itself is tethered to Adobe whereas only the iPhone API is tied to Apple.

"Adobe has repeatedly said that Apple mobile devices cannot access “the full web” because 75% of video on the web is in Flash."

This really is a cheap shot by Adobe in my opinion. First of all, most people that I know with an iPhone wouldn't spend alot of time watching Flash videos on the internet even if it was an option. That's not what the platform was designed for. Secondly, Jobs is right when he says "What they don’t say is that almost all this video is also available in a more modern format, H.264..." The web is moving away from Flash video. With the advent of HTML5, I predict a steep decline in the amount of Flash video out there. YouTube is moving to it and I suspect that most other sites will be quick to follow. It's a better format in general and Adobe is not happy that they are quickly going to be loosing market share because of it.

In addition to Flash video, there is Flash games. The thing is, alot of Flash games on the internet are crap. I can guarantee you that any of the good Flash games on the internet have a comparable game on the iPhone and it will likely be free or cheap. Even if you can't get that specific game on your iPhone, it's not the end of the world, there are so many great games to play on the iPhone and alot of other things you could be doing that are probably more productive.

"Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash."

There isn't much to say here. I don't think this is as much of an issue as Jobs is making it. Although I could be wrong, I don't know enough about the issue to comment for sure. However, I think Jobs is just using this as another argument to just add to his case rather than addressing a major issue.

"To achieve long battery life when playing video, mobile devices must decode the video in hardware; decoding it in software uses too much power."

I think this is just another smaller issue. It's a valid point but again, I think it's just just another argument to add to the list.

"Flash was designed for PCs using mice, not for touch screens using fingers."

Again, another minor point. It's certainly not a reason in my opinion that it shouldn't be on the iPhone, however it is a valid point. From Apple's perspective I can see why it might be a bit of a bigger deal. Apple is all about the user experience and if Flash components bring down that experience, some people will not blame it on Flash but on Apple. It's not Apple's fault, but computer illiterate people will think it IS their fault.

"Besides the fact that Flash is closed and proprietary, has major technical drawbacks, and doesn’t support touch based devices, there is an even more important reason we do not allow Flash on iPhones, iPods and iPads. We have discussed the downsides of using Flash to play video and interactive content from websites, but Adobe also wants developers to adopt Flash to create apps that run on our mobile devices."

This is the strongest reason why Flash should not be on the iPhone. In fact, I think that it shouldn't be supported for this reason alone. Being tied to a 3rd party platform/library can always become a hastle. Especially since Adobe is not known for being quick to upgrade their platform.

If you allow Flash onto your platform, you must now keep your platform compatible with it. If you want to upgrade a feature, you have to make sure that Flash can handle it. Otherwise, all of the Flash applications out there will break and now you have a bunch of angry users. This means that Apple, a company that is known for having fairly quick turnover and software upgrades will be tethered to a very slow moving company. Who do you think is going to be affected? I know for a fact that Adobe won't become a more agile company to keep up with Apple.

I am completely in favour of Apple's choice not to support Flash. I don't think it was a flippant move to simply snub Adobe and keep them away from profits (although this is an added bonus for Apple). It was definitely a carefully thought out move that I think will ultimately make the iPhone a better platform. The gains here clearly outweigh the costs. However, that's my opinion. What do you guys think?

Programming Literacy In Physics

Lately, I am being exposed more and more to the lack of computer programming literacy in the field of physics. Today, most physicists will need to be able to program and will be programming at least 1/4 if not 1/3 of the time. This is because in today's day and age, there is so much data being acquired that you have no choice but you use a computer to process it all. And yet, there are far too many physicists who don't know anything about programming. They know enough of the basics to be able to write their algorithms, but they are nowhere near the quality that they should be.

This rant was triggered by a conversation I had with a colleague yesterday. I am currently developing software to sort experimental data coming from our data acquisition system in real time. I was telling him that I had originally designed the code to be implemented using threads and so alot of my design choices had been due to the fact that I was trying to avoid race conditions wherever possible (it turns out that the algorithm is fast enough that we don't need to thread it but that's beside the point). At this point in the conversation, he indicated to me that he didn't know what a race condition was. I was a little disappointed but I tried to explain it to him using an example.

I started off by saying, "Suppose you have a binary tree..." At that point he chuckled a little bit and I could see the confused look in his eyes; he didn't know what a binary tree was. I then decided to explain to him how a binary tree works. I described to him how the complexity of searching a binary tree is O(log n) as opposed to a linear search which is O(n). Again, he chuckled because he had no idea what I meant by complexity. At this point I was almost in shock. How can a person who has to deal with C code all day have no idea what computational complexity is. However, this is no an isolated occurrence. A few days ago, I had to show my boss that there is a pre-implemented quick sort algorithm in the C standard library. If I hadn't shown him that it was there, he was most likely going to implement a bubble sort.

Unfortunately, I don't see this problem going away any time in the near future. The problem stems from the fact that most physics programs include only very basic computer programming classes. At the University of Windsor, you have 2 choices when it comes to computer courses. The hard option is to take Introduction to Algorithms I & II. The easy option is to take C for Beginners and then Introduction to the Internet.

First of all, there should be no easy option that lets you off taking a course on how to write HTML and a C class that teaches you nothing except perhaps what a for loop is. In a field where programming is a mandatory part of the job, that sort of knowledge is insufficient. Second of all, Introduction to Algorithms I & II are woefully lacking in the number of algorithms that they teach. I think the most complex algorithm we wrote was a bubble sort. This is not a problem that I can attribute to the physics department, it is the fault of the computer science department. However, in the undergraduate physics curriculum, there is no opportunity to do any sort of scientific computing at any point. Thus, unless you are self-motivated enough to do programming yourself, you get no experience writing any sort of code except in those two VERY basic programming courses.

From what I have heard from other physics students at other universities, this problem seems to be the norm more than the exception (even though it comes in may forms). Some students will get a scientific computing class, but they have to write everything in FORTRAN. This stems from the fact that alot of physicists did most of their computer 20 years ago in FORTRAN and never bothered to learn a new programming language. Just to give perspective, FORTRAN was originally run on punch cards. Others are writing all of their code in Maple (a computer algebra system for solving equations analytically). This is fine if you are teaching some mathematical analysis however you will never learn about performance concerns and efficiency since these issues are handled by the Maple backend.

If it were up to me, I would make it mandatory that all physics students learn about basic complexity analysis and simple sorting and searching algorithms. If you teach them, I can guarantee that half of the scientific computing code out there would probably reduce their run-time by about 25% if not 33%. This wouldn't be a hard change, simply make them take a computer science course teaches these two concepts. Also, if you are going to teach a scientific computer course, do NOT teach it in FORTRAN. I would say the best language to teach that sort of course in would be Python, either that or C/C++.

Either way, I think this is one of the biggest problem areas in all of physics today. If you don't know even the most basic of algorithms, you are doomed to implement inefficient, limited solutions that will leave you re-inventing the wheel every step of the way. Also, this type of code becomes near impossible to fix because it is typically nigh unreadable to even the program's author.

C Function Pointer To Class Methods

My current project at work involves me writing an application to sort data coming from a server and then repost it to the server in a shared memory buffer. The application is being developed in C++ but the server application was originally written in C. As a client of this server, I have to register a series of callback functions for various event transitions that occur on the server. The problem that I had was that I needed to pass a function pointer to an object method. However, you can't just pass a pointer to the method because you can't call an object method without an object to call it on. There are two solutions to this problem, the first is to pass a pointer to a static method. The second, is to set up a wrapper to the object method.

In the first method, you would do something like this

class A{
static void memberFunction();
};

int main(){
registerCallback(A::memberFunction);
}

This will work in most cases, however depending on your compiler it may not work. The reason is that to pass a completely standards compliant function pointer, you need to pass a pointer to a non-member function with C linkage (using an "extern C" call).

The second option is to use a wrapper function. Simply wrap the method call in a non-member function and then pass a pointer to that function. An example of this would be

class A{
void memberFunction();
};

A* pointerToCallbackObject;

void wrapperFunction{
pointerToCallbackObject->memberFunction();
}

int main(){
registerCallback(wrapperFunction);
}

Cool Linux Tools

Lately I have been experimenting with a couple tools for my Linux machine and I figured I would share them with you guys.

feh This little app is a command line image viewer. If you need to look at a picture quickly and don't want to open up a heavy weight image viewer you just run "feh [filename]" and you can view your picture, close it with Ctrl-C and then be on your way.

convert This program is great for when you need to quickly convert a picture between file formats. You call "convert [input] [ouput]" and have a quickly converted image. It's also very powerful and has alot of extra options if you need high quality copies or want to resize the photo. So far, I haven't come across an image format it can't parse.

screen I've just recently started using screen when running my myriad of ssh terminals at work. I typically am ssh'ed into about 4 or 5 different machines at a time and will typically be running about 3 processes on each of them. I can open up a separate tab for each process and that works, however it really clutters up my terminal window. Also, the wireless connection in the building can be spotty at times and so I will sometimes end up getting disconnected from my ssh connections loosing all of those processes which is a serious pain. This program acts as a sort of terminal multiplexer, you can run up to 9 different terminals from a single window which is fantastic, but not the coolest part. The most fantastic part about this application is that you can disconnect from the screen session and then reconnect later, leaving all of those programs running in the background. You can even share the screen session allowing multiple users to access it.

Edit: Change some of the text that was getting parsed as HTML tags.

The Beginnings Of A Blog

Hello, everybody. I guess this would be my first post. A few of my friends have really been getting into blogging lately and I've felt the need to jump on the bandwagon.

Seeing as I am a programmer and physicist, this blog will mainly be about those two topics. You can expect to see things like code snippets, projects that I'm working on, and news that I find interesting from the worlds of programming and physics. From time to time, I might also go on a rant about something that I'm feeling passionate about at the time. It could be politics, philosophy or anything else that I'm thinking about.