List of useful Python libraries

If you are a .NET programmer, then you find Python a bit tough. Reason? Python does not include library for each and every operation possible in this world. You may have to work around to find the necessary packages, download them and continue with your development.

Python’s standard module list has a finite number of entries as opposed to .NET    ( I use .NET at my workplace). Here this is an attempt to collect all such libraries which are outside standard modules, which you might badly need for your development. Many of them are extensions or wrapper packages for already existing libraries.

1 ) scapy

This is a library for  TCP/IP stack wherein you can have full control over the lowest detail of the Packet that leaves your computer. It supports many protocols like ETH, IP, ARP, ICMP, TCP, UDP etc. You can create custom TCP/IP packet and send it to any host. Typical implementation is ARP Ping, ICMP Ping.

Experience: Tried. Works perfectly. Havn’t stumbled across any bugs as of now.

2) soaplib

Used for creating lightweight web services. As the page says, it comes with a client and server built in and on-demand WSDL generation.

Experience: Havn’t tried. Heard about it’s existence.

3) mysql

Uh? Do I really have to tell what this is actually. I hope everyone knows.

Documentation for python-mysql

Experience: Obviously! Obviously! I think I should remove this line.

4) aubio

Stating directly from it’s site – “aubio is a library for audio labelling. Its features include segmenting a sound file before each of its attacks, performing pitch detection, tapping the beat and producing midi streams from live audio. The name aubio comes from ‘audio’ with a typo”

Experience: None. Presently in To-Do List.

5) Beautiful Soup

BeautifulSoup is an SGML parser which is highly robust and doesn’t die straight-off even if you give it poorly formed data. To make it scream and die all you have to do is to give something that isn’t SGML at all. It even has a parser class named BeautifulSOAP which is used to parse SOAP message (as the name applies). It even has a class named ICantBelieveItsBeautifulSoup. Sounds stupid? Who cares as long as it does it work.

Experience: Tried when I saw Anomit using it. Need more experience as I have lost touch as of now. Never tried BeautifulSOAP.

6) python-clamav

It is pending in my To-Do list. Will start working as I get time.  Check a small tutorial

Experience: No! Read the line above.

7) python-crypto

Presently in #1 position of To-Do list. Sounds just too promising. Hope it is as I thought it to be.

Check the API and it’s general overview

Experience: No

8 ) django

Now if you don’t know django – Go shoot yourself or read about it here if you somehow survive.

[ As pointed by Anomit, it isnt a framework, but library is a general name I have used for the title ]

Experience: Obviously!

9) gd

I have used GD a lot in PHP, but hardly on Python. GD is simpler than ImageMagik (never used) as people told me. Hope to use this library if I ever require.

If you ever require the documentation, head yourself to this page.

Experience: Not used in Python, but in PHP

10) gmp

GMP stands for GMP Multi Precision and gmpy is a python wrapper over it. Though you might not need it in Python, but if you are coming from C background, this might be a familiar name.

Experience: Normal, not an expert

11) python-jabber

Python-Jabber is a python module which implements jabber instant messaging protocol. Check out the documentation and a funny example .

Experience: Little experience. Not much. After all it doesn’t look so tough, so will sit down for a hacking session,

12) python-irclib

I encountered this library when I was searching more on python-jabber library. This also falls in the category of real-time messaging. The problem I can see is that there is no documentation. How to proceed? Use dir() and inspect module extensively?

Experience: Kidding? Please show me the documentation. I don’t have more time for hacks as I did with scapy.

Till now ,I had kept this list for my own reference. Many more required libraries are missing. If you have any more in mind, please mention it. I would be glad to add it.

Checklist for fresh Ubuntu installs

Now don’t bash me just for installing proprietary and restricted software on my computer. I have no other option for the time-being. I am too a free software evangelist.

Each time I (re)install Ubuntu on my laptop or home PC or on a friend’s laptop, I always forget some or the other software which I have to download at a later date. This is sometimes troublesome if I don’t have net working at my home and take the laptop to a wired workplace to download the packages. Here are the packages which I usually download and you would always find it as a part.

This includes many proprietary softwares as well, if you know a good replacement, please do take pains to inform me about it.

Runtimes

  • Adobe Integrated Runtime (proprietary)
  • Java 6 (free)

Applications

  • Twhirl (proprietary)
  • Terminator (free)
  • xChm (free)
  • Inkscape (free)
  • TweetDeck (?)
  • K3B (free)
  • Scribus (free)

Networking (incl Internet)

  • Thunderbird (free)
  • Epiphany (free)
  • FileZilla (free)
  • WireShark (free)
  • XChat (free)
  • Opera (?)
  • KTorrent (free)
  • Bluetooth OBEX Server (free)

System Tools and Softwares

  • Yakuake (free)
  • VirtualBox OSE (free)
  • Konsole (free)
  • QGRUBEditor (free)
  • APTonCD (free)
  • Wine (free)

Programming (Editors + SDK)

  • Geany (free)
  • Glade Interface Desinger (free)
  • Komodo (proprietary)
  • Netbeans (free)
  • Qt Designer + Qt Creator (free)
  • Qt SDK (free)
  • GTK+ Development Headers (free)
  • Leafpad (free)
  • Emacs (free)
  • Amaya (free)
  • QT4 Linguist and Assistant (free)
  • MySQL (free)
  • Apache (free)
  • PHP (free)
  • SQLiteman (free)

Multimedia

  • VLC Media PLayer (free)
  • Amarok (free)
  • Banshee (free)
  • Last.fm Scrobbler (free)
  • Miro Internet TV (free)
  • RealPlayer (proprietary)
  • Gstreamer Codecs (proprietary) (restricted)
  • Xine (proprietary) (restricted)

Configuration

  • Dust Theme (free)
  • Nautilus Action Configurations (free)
  • Nautilus Open Terminal (free)
  • Compiz Config Settings Manager (free)

Final Semester Project

I have been working on my Engineering Final Semester Project. We are required to make one big project and the time frame alloted is one complete semester! As everyone expects from me, I am using Free Softwares and Open Specifications for my project.

The technologies being used in my project are: PHP, MySQL, Python, SQLite and XMPP for the initial. More technologies can come into picture as and when required. So lets’s start with what am working on? Its a software development platform like Launchpad. Launchpad had their own source control management software(bazaar), then why cant I? My project has sub-sections as Source Control Management resembling a lot with subversion. I have a client and a server module coded in Python.

The data on the server is being stored using MySQL database and the client uses SQLite as database software. Apart from this we also intend to have a issue tracker which will be written in PHP and MySQL running over Apache. All the modules of this Software Development Platform would be tightly coupled with single login and User Access Control.

I am also into thinking that we need to have some IRC kind of real-time chat room where developers can collaborate to work together. Oh Wait! We can use an XMPP bot to keep a watch on the server. XMPP bot can be another frontend to handling your work just from an IM client. Developers should leave status messages related to specific projects so that visitors can know what work is being undertaken by the developers right now.

I am having a gala time working with Python, its an awesome language having short,precise syntax. If you have a good lightweight editor with basic auto-complete, you don’t even have to write anything. 🙂

I would be posting soon the experiences with Python and SQLite. Keep a watch!

Open Source is a Viable Business model

This is probably the most debated and inquired question I have faced till date. Is business model built on top of Open Source softwares viable? Newcomers to this concept find it difficult to digest that a company can survive by opening up the source code of its product. There are also many learned people who still disagree with this fact. This discussion springs up in many forums, IRC channels and mailing list which sometimes turn into a flamewar.

Normally when we say Open Source and Free, the thing we visualize is “free of cost software”. Free always means Free as in sense of Freedom, otherwise stated. None of the FOSS licenses say that you can’t make money out of your software. There are many companies making highly popular Open Source softwares and are always in the forefront like Red Hat, Novell and Mozilla.

Where is the Money?

Now the question comes, where do you make money? One way is to sell the software(intellectual property), other way is to sell the support. A person can download the source code, strip off the Trademark stuffs like name, logos etc and use it without paying a dime. In this scenario, the second way becomes more viable. The reality is that second way is the most famous source of income of these companies(say XYZ). If you have a reputed company(say ABC), you won’t hesitate to pay extra fees for support to keep your business running on the track. You may even consider extending the support if the first support period expires. Sensible companies like ABC care more for support rather than cost of the software they buy. Even if the software is awesome and support turns out to be crap, you are in a quicksand, more you try to come out, more you go in. If the support is excellent, then you would always be assisted by the tech support even if the product is not upto the mark.

People quote that who would anyone give away their hard work for free? I ask, why reinventing the wheel? No body forces you to open the source code, its all upto you to act on their recommendations. If one opens up the source code, people may be able to read the code, find new bugs, performance glitches and even make available the patches. This is how FOSS world works. You can even sell the patches version of your software. It all about sharing and caring.

Real Examples

Let’s have a look how Mozilla makes profit. It gives away its product for free, AFAIK all its products are free. What is its source of income? Check out the search bar of firefox, you would find that the default search engine is set to Google. This is a major traffic source for Google because who cares to change to default search engine? At aleast not me! Google pays a hefty amount for just setting the default search engine, what a trick! Mozilla is all submerged with $$$ these days and its revenue is increasing day by day.

Many companies give away their products for free and charge for support. One of the best example is MySQL AB which was acquired by Sun rcently. MySQL is dual licensed. If anyone wants to use it, they can download the community server and even use it for commercial purposes. No technical support is attached with it. The other is the Enterprise Server which comes with full technical support and even extra propriotery tools if you wish to buy.

Business Prospects

This is the time of global recession. Companies are handing pink slips to many of its regular employees since they don’t have enough money to sustain. A big amount of their budget goes in buying licenses and updates to already highly priced softwares. The money saved by moving to Free (as in Beer and Freedom) softwares can be enormous. The initial cost can be even more due to training the employees, but its the long run which matters the most. Such transition can be taken during merrier times which makes a firm ready for the bad days.

If one wants to start a new business, then highly charged proprietary softwares can be a main hurdle in the beginning itself. Investing a heavy capital can be a deterrent factor for smaller players into the field. If such new firms use Free Software then the initial capital is greatly reduced. More the number of players in market means more competition and its always the customers who benefit from it.

Concluding

I think six paragraphs of explanation is enough to explain this concept, if one still can’t understand, the best place is a flamewar on IRC or mailing lists.

Running PHP Applications on Linux

Overview:

I usually find that lots of poeple find it quite difficult to install any application made in PHP and MySQL on their Linux boxes. They find it a bit confusing how to configure the settings. How to install the LAMP stack. Where to put the PHP applications code. I was quite inspired to write to this after having I saw people facing problems on Ubuntu Forums. This is the one which I dealt with.

Sometimes spoon-feeding becomes important,but only for the first time.

Scenario:

A newbie who knows just the basics of Linux. He/She wants to run a PHP application like wordpress / phpBB / osCommerce on their computer.

The distro is assumed to be Ubuntu, though it doesn’t change much for others too.

Lets Start:

Step 1: Getting the LAMP stack on your system

You first need to install LAMP on your box, Installation is quite simple. Just get the following packages using the command on your terminal ( Applications> Accessories>Terminal)

sudo apt-get install apache2 php5-mysql libapache2-mod-php5 mysql-server

While installing MySQL, it might ask for set up the root password,if it doesn’t ask, then continue.

For more information check this post here .

Step 2: Download the PHP application

You need to download the PHP application which you want to run over the webserver. Examples are WordPress, phpBB, osCommerce, MediaWiki etc

Get them and let it be in the folder where you have downloaded it.

Step 3: Moving these software to the Apache’s root directory

Apache’s root directory is /var/www (/var/www/html). This is owned by the user www-data which is a minimal user created just for running apache and web servers. A normal user cant write to this directory.

Now here is the solution of how to copy the data to this directory. This method can be used for any directory which the normal user does not have write access. The concept behind the method is to start an instance of nautilus ( explorer) with root (superuser) privileges. For this follow the isntructions

* Press Alt + F2

* A Run daialog box appears, over there type gksudo nautilus /var/www

* You will be asked for your password as you are elevating your privileges

* Nautilus opens up,now copy the tarball/zip file from the download location to /var/www

* Untar the archive by Right Click > Extract Archive

You may have the package in the folder say /var/www/wordpress. We are supposing wordpress as an example.

Step 4: Setting up username and accounts in database

Now you need to have a create a new database for the software you intent to work on. For this open MySQL from the terminal. If you had been asked to set up the password at time of MySQL installation then use this command

mysql -u root -p

and then enter the root password which you set up

if you were not asked to set up root password, then use this command at terminal

mysql -u root

After this is done you will be dropped to mysql prompt. Create a new database using this:

create database foo;

type quit to exit.

Step 5: Change the data in the config file.

Hunt for file which may have name such as config,settings etc etc. One you locate it,open to find the area for entering database settings. The settings should be:

Hostname: localhost

Username: root

Password: <the password you set up>

Database name: foo (or whatever you created)

Step 6: Access it via browser

Now point your browser to http://localhost/wordpress

You may get the basic installation thing or whatever stuff you need.

Additional Stuffs:

You can be asked to make some locations writable. There may be instances like the software cant write to a specific folder or file. It can be used for cache or any such temporary stuff. Then you need to change the permissions. Simply give write permissions to the user www-root. However this situation may be unlikely. e..g we have to change the permissions of folder /var/www/wordpress/cache

sudo chmod -cR u+w /var/www/wordpress/cache