[Update: Please see at the bottom of this post for a link to an improved version of Padma.]

Anandabazar Patrika (ABP) and Bartaman Patrika (BP) are two (among big four) well-known Bengali news papers that are published from West Bengal, India. In the Internet era, their online versions are not just a matter of convenience rather the only route of access for many of us. Unfortunately, their online versions continue to live in the past by using non-standard, ancient dynamic font technology instead of upgrading to standard Unicode.

The worst part is that to view their website you need to have Internet Explorer installed in your machine. So if you are Linux, Mac (or any non-Windows) users then you are left at your own.

Fortunately, there is now a simple way for Firefox users in Linux and Mac to read these websites using a Mozilla extension named Padma by Nagarjuna Venna and his team. To get Padma working, (a) you need to have Unicode Bengali font (Linux users may already have one. Mac users can get one from Ekushey), (b) you need to have Firefox (version 3 is recommended for Linux but must have for Mac), and (c) and you need to install Padma.

Padma can transform given non-standard encoding to standard Unicode on the fly. Of course, for Padma to work, it must know the font-encoding of the particular website.

As it turns out, I wrote support for ABP in Padma more than a year ago. My job was made simple by an earlier CGI program by Tanmoy Bhattacharya who had already decoded font-mapping for ABP. Couple of months ago, I also added support for Bartaman Patrika in Padma. So, courtesy Tanmoy’s font-map decoding, latest version of Padma (0.4.13) supports both ABP and BP.

There is a known issue of incorrect rendering of Bengali Matras in certain situations. See for example Runa-Sankarshan’s photostream here. Many of these were due to a simple bug and has been fixed in the latest version (0.4.13). However, fixing of the remaining requires significant changes in Padma. ABP and BP both use three different fonts simultaneously. Most ligatures often come from 2nd and 3rd font whereas Matras come from the 1st font. Padma transforms each font separately and doesn’t merge these different fonts elements into a single element. This leads to the incorrect rendering which is hard to solve without changing the core of Padma.

The Bigger Issue however is the need for Padma itself. I tend to agree with the concerns expressed by Sankarshan in a discussion thread here. The real question is then how long are these websites going to keep themselves confined using their own non-standard encoding?

This led me to wonder: don’t their technical staffs realize what they are missing by not upgrading to Unicode? Firstly, by upgrading to Unicode they could readily expand their current user base. Secondly, the use of Unicode will make their contents search-able in search engine like Google. This could lead to additional search-engine generated revenue for them. The number of Bengali internet users is going to increase in coming future, and a significant portion of new internet users will be coming from the interior part. Undoubtedly, many of these users will be more comfortable in searching using Bengali keywords. Thirdly, by continuing the use of non-standard encoding, they are piling up their archive with non-standard contents which would require a big effort by them to bring into standard form. So, in my humble opinion, it would be prudent decision for them to upgrade their website to use Unicode sooner than later.

Nevertheless, there is now a positive sign that Star Ananda, a sister group of Anandabazar Patrika, has started using Unicode (though their defined “charset” doesn’t say so) for their Bengali website. I hope, this marks the beginning of change.

Update (May 9, 2009): Please see this post for an update on the above mentioned incorrect rendering issue.

If you are familiar with LaTeX and interested in writing Bengali documents (or any other Indic documents) using it then you might find this post relevant.

In brief, LaTeX is a type-setting system which is widely used in scientific communities for preparing technical articles.

Here, instead, I will be interested in writing Bengali documents using LaTeX. The first Bengali typesetting system using LaTeX was the well known Bangtex package by Palash B. Pal. Bangtex does produce good quality documents.

However, main inconvenience in using Bangtex package is its input method. One needs to follow a specific transliteration scheme. This suffices if you want to create PS/PDF file at the end. However if you also want to create html pages for the same documents using latex2html then you are out of luck.

Now wouldn’t it be great if you could type your Bengali (or any other Indic) documents using standard Unicode in LaTeX as you would do with any other application?

Here comes the new revolution, XeTeX, a new type-setting system based on a merger of Donald Knuth’s TeX system with Unicode and modern font (OT/AAT) technologies. This means you no longer need to have special metafonts for your language to be present in the system. Instead you can use standard OTF/TTF fonts those are available in your system, for LaTeX as well.

No more talking… here goes the image of output PDF file.
Bengali sample document using XeTeX

Above document is created using the following latex source file


% This file is a sample tex file to illustrate use of
% XeTeX in typesetting Bengali document.
% License: GNU FDL: (C) Golam Mortuza Hossain, 2008
% Define fonts that you want to use
\font\serifbb="FreeSerif:script=beng" at 24pt
\font\deffont="FreeSerif:script=beng" at 14pt
\font\mukti="Mukti Narrow Bold:script=beng"

% "script=beng" in above lines ensures that appropriate
% complex text rendering engines are used for proper
% rendering of Bengali.

\title{\bf\serifbb পাগলা দাশু}
\author{\bf\serif সুকুমার রায়}

\deffont %Default font used for the document

আমাদের স্কুলের যত ছাত্র তাহার মধ্যে এমন কেহই ছিল না, যে পাগলা দাশুকে
না চিনে। যে লোক আর কাহাকেও জানে না, সেও সকলের আগে পাগলা দাশুকে চিনিয়া
লয়। সেবার একজন নূতন দারোয়ান আসিল, একেবারে আনকোরা পাড়াগেঁয়ে লোক, কিন্তু
প্রথম যখন সে পাগলা দাশুর নাম শুনিল, তখনই সে আন্দাজে ঠিক ধরিয়া লইল যে,
এই ব্যক্তিই পাগলা দাশু। কারণ তার মুখের চেহারায়, কথাবার্তায়, চলনে চালনে
বোঝা যাইত যে তাহার মাথায় একটু 'ছিট' আছে। তাহার চোখদুটি গোল-গোল,
কানদুটা অনাবশ্যক রকমের বড়, মাথায় এক বস্তা ঝাঁকড়া চুল। চেহারাটা দেখিলেই
মনে হয়---

\vskip 10mm

% Specific font can be used for any part of the document
{\mukti বিঃদ্রঃ এই বাংলা পাতাটি লেখার জন্য XeTeX ব্যবহার করা হয়েছে।}


XeTeX is available as standard component of TeXLive distribution. If you are using latest (K)ubuntu then you can

sudo apt-get install texlive-xetex

to install it. To test above sample tex file you can cut-n-paste to a file say sample.tex then compile it using

xelatex sample.tex

If everthying goes fine then you should get sample.pdf as your output document. In case you need to edit the latex source file, you can use any editor of your choice like kate, gedit or even vim. Of course, you can also use specialized LaTeX editor like kile (KDE Integrated LaTeX Environment) where you can view output document simply in one click!!

Update (Jul 8, 2010): If you are facing “double-quote” mixed-up problem that causes Latex compilation error while copy-pasting from the browser then please download the sample.tex file here.