Tag - ocaml

Entries feed - Comments feed

Last entries

Tue 26 Jan 2016

WE programming project: XKCD keyword index

With this post I'm starting a new kind of article: WE programming project. It's objective is to give ideas of programs that could be written over a WE, probably a lot more or lot less depending on your skill and willing to extend the idea.

Here is the first project proposal: an XKCD keyword index.


You probably known XKCD, "a webcomic of romance, sarcasm, math, and language". If not, I recommend it's reading. :-) All the XKCD comics are freely available, through an URL, e.g. "Asteroid" is available at URL https://imgs.xkcd.com/comics/asteroid.png.

Sometimes, I would like to illustrate a slide or blog item with one of XKCD comics. I know it exists, I saw it but I can no longer find it! So frustrating! So the main idea of this WE programming project is rather simple: make a keyword index of XKCD comics so that one could look for a comics over a theme or idea.


Here is some requirements for this project, feel free to adapt to your needs or your own ideas: ;-)

  • It is a web site on the Internet that displays each XKCD comics (with proper reference to original web site) with associated keywords
  • One can search comics on a set of keywords
  • One can associate keywords (a-z- character set, space separated list) to a given comics
  • Already given keywords are suggested
  • No need of account by default, everybody is free to add new keywords (for easy contribution)
  • Moderators (with dedicated account) can make previously added keywords public, can remove keywords
  • No use of database (for easy installation), keywords are saved into a simple file
  • One can download the whole association of keywords to comics (e.g. as a JSON data structure)
  • There is an API to control the web site (for use by others and integration into other sites)

Needed programming technology

Any programming language you want which have a well designed web framework: Ruby with Rails, Python with Django, Javascript with Meteor, OCaml with Ocsigen, ...

If you tackle this project proposal, let me know! ;-)

Sun 26 May 2013

Issues with distributions, not only a Debian specific problem

Some time ago, I blamed Debian for not looking enough at its users. Apparently, I'm not the only one to make such remarks. Ingo Molnar, a famous Linux kernel developer working for Red Hat made similar remarks. He even proposed some technical solutions.

Other solutions are already available. For example, for OCaml specific software, OPAM, an OCaml specific package system, seems very interesting to work around the freshness issue of OCaml Debian packages.

I'm not sure Ingo's answers or OPAM are THE answers, but they at least open interesting perspectives. Handling of security and licensing issues are very difficult and not yet solved. Distributions are making an heavy work of integration that is currently not done by anybody else.

It is nonetheless refreshing to see people thinking at those issues and proposing new solutions. I find Ingo's proposal of sandboxing, flat dependencing and not forcing people to upgrade very interesting. If you read French, you can also read this LinuxFR article that makes a small review of current proposals.

Sun 27 Jan 2013

The failures of Debian (and its derivatives)

I am a long term Debian user. I have used Debian since probably the mid 90' up to now. I use latest Debian stable on my personal server or at work, and Ubuntu on my desktop and laptop machines at home. Using it does not mean I am entirely satisfied with it to say the least. I think the distribution is not making enough efforts for its users and I'll try to explain why. Of course, YYMV. ;-)

  • Debian packaging is made for developers, not users. Debian has too many packages. To take just one example, the OCaml compiler is packaged into the main compiler, its libraries, the native compiler and the byte code one, etc. Why so many packages? If I am an OCaml developer, I want all of them so why do I need to manually find (the naming is far from being obvious, at least for beginners) and install so many packages? I have heard of several reasons: it allows to factorise common parts between the different binary architectures, it allows to install the command line only parts without the graphics parts, it allows to install only what the user wants, etc. For me, those reasons are just plain wrong. We have more and more disk capacity on our machines so disk usage is no longer a limitation. The package should be able to dynamically activate the graphic parts automatically if the X server is available. And the factorisation of shared parts between Debian architectures should be done on the servers storing the packages, not trough the packaging system.
  • Debian has out-dated software. Debian Wheezy is about to be released and it will have GNOME 3.4. But GNOME 3.6 is already out and GNOME 3.8 is on its way. And I am taking GNOME as an example, it is the same issue for lot of software within Debian. I have heard this is for stability issues. But software packaged in Debian is already stable! It should take 10 or 15 days to integrate a new software into Debian stable, not months or even years (the time between successive stable releases). I acknowledge that some packages have complex interdependencies between each others. For example, when a new release of the OCaml compiler is out, one needs to recompile all OCaml packages. But this constraint is only for OCaml packages. Why should I wait for a new stable version of Debian to get the newly released OCaml compiler? For me this sounds just plain wrong.
  • Nobody uses plain Debian stable. Debian developers are using unstable. Debian users are using Debian stable, but enriched with backports because of out-dated software. Or derivatives like Ubuntu. The fact the Debian developers are not using what they recommend to users is bogus. I know they do that for technical reasons, but that should not be a good reason. Debian developers should use what they are providing to their users, except maybe for a few packages they are working on.
  • There are too many dependencies between packages. The dependency system of Debian is a nice piece of work, it was ahead of its time when it was created. But the use of dependencies has been perverted. The dependencies are manually computed (thus source of errors and bugs) and at the same time any software can write to about any part of the file system. Due to too many dependencies and lack of clean interfaces between sub-systems, the installation of a recent user software can trigger a ton of packages down to a new kernel or libc. Why is it so? I think the sub-systems of Debian (e.g. the X graphical infrastructure, the kernel and base C libraries, the OCaml system, ...) should be isolated the one from the others. It would allow them to be updated without waiting for the others. Having dependencies between 29,000 packages is just not scalable. It is even more true if those dependencies are manually computed.
  • Debian packaging is lacking automation. I am a developer. Debian packagers are developers. It always astonished me how much manual work should be done to make and maintain a Debian package. All developers know that if they want to be efficient, they need to automate their work as much as possible, so as to be able to focus their manpower on the complex parts. Everything should be automated in Debian packages, starting from a minimal description file. Automation should be the default (package creation, test, static analysis, ...), not the exception.
  • One cannot install simultaneously several versions of the same software. As a user or developer, I want to use the stable version of a piece of software and maybe the latest stable version that just have been released in order to do a few tests or check that my software still compiles with the new shiny compiler. Debian does not allow me to do that. And if I install a newer package, downgrading to the previous version is complex and error prone.
  • Debian is undocumented. I am not talking about the installation guide which is nice, I am talking about the modifications made to software by Debian developers. Doing modification on the "standard" (for the software) way of configuring or using it has always seemed suspect to me, even if I agree that having harmonized configuration is a real advantage (all configuration files in /etc for example). But all of those modifications should be documented in README.Debian file. To take an example, the last time I tried to install the dokuwiki Debian package, I was unable to configure it! The way to add new users had been changed compared to a regular dokuwiki (the web interface was disabled), and those changes were undocumented. It should be a release critical bug! Without proper documentation, the user cannot use the software. And, of course, the reason behind those changes should be questioned, even for security reasons (a very secure but unusable software is superfluous. Security is a trade-off).

So, why I am complaining? Why I do not become a Debian Developer, so I can fix it? Because a single developer is not going to change the root causes of those issues. They need a massive development effort, or at least a massive acknowledgement by the Debian Developers. And I don't have ready-made answers to those issues (even if I have some ideas to solve them).

Is the grass greener in the other fields? I don't think so, or at least I am not aware of it. I like Debian for is community approach, its focus on Free Software (even if it is sometimes imperfect) and the wide range of software packaged in it (the OCaml packages are numerous for example). I just hope that the whole Debian community will focus on more user related issues in the future.

Fri 02 Dec 2011

A draft of library to handle physical units in OCaml

When writing a computer program that interact or handle with the physical word, a common need is to manipulate physical units like meter, second or km/h. Most of the time it is up to the programmer to ensure that the physical units are correctly used, even if there are rules to check them (divide only meters by meters, ...). All physicists are routinely checking their equations with those rules. Why not programmers?

So here is a draft of OCaml library to do just that: computations on physical values while ensuring physical units rules are followed (for example, you only add meters to meters, not meters to seconds). Here are the SIunits.mli and SIunits.ml files.

The use of the library are quite simple:

  • A set of functions to create values with units from simple floats. For example meter 1.0 (or simply m 1.0) creates a value containing 1 meter;
  • A set of utility functions to handle SI prefixes like kilo, to normalize a value (i.e. to remove the scaling factor) or to print it with its units. For example, to enter a length of 160 km one would write kilo (m 160.0);
  • A set of four operators, +!, -!, *! and /! to do basic computations on those values. For example, to enter a speed of 160 km/h, one would use let speed = (kilo (m 160.0)) /! (hour 1.0). To compute the distance travelled at previous speed during 120 s, one would do let distance = (normalize_si_value speed) *! (s 120.0) and the program can print 5333.333333 .m;

Internally, I have followed the proposal of "Checking SCADE Models for Correct Usage of Physical Units" paper by using a vector of 8 units: 7 base units (following those defined in the SI system) plus one power of 10 unit for the scaling factor (to handle kilo, mega or more exotic units like hour). In fact, to be really exhaustive one would need some additional units but the current code is really a draft. Then, the rule for addition or subtraction is to check that the units are identical, while for multiplication and division the units of the two numbers are added or subtracted. See the above paper for details.

There is a lot of work to transform this draft code into a real, usable and exhaustive, library:

  • Power and root operators;
  • Comparison operators;
  • Decision operators;
  • Dimensionless units;
  • Absolute or relative values;
  • More prefixes and exotic units;
  • And of course of lot of tests for above code.

Right now, I do not intend to extend the current code but if other people are interested, I could work on it and open an OCaml Forge project. If there is a need, let me know.

And now a question for the OCaml experts: would it be possible to replace the dynamic checks currently implemented in this library by static checks using OCaml type system (for example using Phantom types)? I think this is not possible: the base units are like arrays of the same size, you need a more expressive type system to express constraints on them (like Coq's Dependent types). But I would be pleased to be shown wrong. :-)

Sat 11 Sep 2010

Literate programming: where we are and where we should aim at

Book and paper (CC-BY) Literate programming is an old idea: Donald Knuth invented it during the 70' to write TeX, the typesetting system.[1] The main idea of literate programming is to document a computer program while writing it by intertwining pieces of code and explanation. In other words, instead of writing a program, one writes a book that documents and contains the program. Then, two programs are used to extract the (i) the compilable code and (ii) the book ready to be printed.

The main philosophy behind literate programming is that:

  • One should document his/her program;
  • By putting program documentation very close to program text, we improve significantly the ability to update the documentation when the program is changed.

Where we are

Several literate programming systems now exist. Most of them are grand-child of the original Knuth's WEB system. They can be either generic, like noweb, or specifically tailored to a programming language like OCamlWeb for OCaml. The main idea of those tools is to write a LaTeX document and put pieces of code in it. For example, I have used noweb to write demexp, a special voting system for massive direct democracy. The OCaml code of demexp can be read like a book.

The main issue with those tools is that you need to write a LaTeX document. You cannot use your preferred (or mandatory) IDE to edit the code and it is rather cumbersome to use in a Windows environment.

That's why we have developed Lp4all. LP4all is a literate programming system where documentation is put within the language comments, using a special syntax close to a Wiki one. From those special comments one can generate web pages, like those for Lp4all itself.

However I'm not using Lp4all!

I'm not using it because, in my last document at work, I needed all the power of LaTeX:

  • Advance LaTeX packages to make special figures, add hyperlinks to the generated PDF document, ...
  • Use LaTeX ability to create special counters for dedicated cross-references;
  • Ability to pretty-print a relatively confidential programming language;
  • etc.

So, on one side, I need the flexibility of Lp4all, its ability to integrate within any IDE or development environment. On the other side, I need a complex documentation system, giving to me all the tools to fine tune my documents and produce quality documents. Moreover, Lp4all is limited to a single program while I need to generate various documents from a single source.

Where we should aim at

(or at least some ideas about it ;-)

One simple solution to above dilemma would be simply to add LaTeX capabilities within Lp4all. Some tools like ocamldoc (the OCaml equivalent to JavaDoc) is following this approach, with extensions to mark parts using the LaTeX syntax. This probably the most practical approach. After all, Wikipedia is using the LaTeX syntax for mathematical formula.

However I find that this approach is not satisfying.

First of all, LaTeX is a very weak system (a package can break another package) and is very complex to use when you want to extend the system. It would be much simpler if we could have a documentation system build upon a sane language.

Moreover, to really exploit the full potential of literate programming, one needs to produce and track dependencies between several documents. For example, if I modify the specification, I need to modify some parts of the code. Inversely, if I change my code I need to know which part of the "specification" have to be updated. This tracking between pieces of code/documentation goes to test, user documentation, code documentation, test review, Q&A documents, GUI design, mathematical scripts describing an algorithm, formal models to check the software, etc. And more importantly, if I modify a piece of this code/documentation, I need a list of all document parts I need to review and possibly update, a kind of make on steroids.

A third point is that one needs more graphical interfaces: it would be easier to graphically link a piece of code to a document paragraph describing it than create a LaTeX reference and reference it later in the document. Moreover, documents nowadays use graphics extensively and we need to link pieces of code/documentation and some graphics. For example, it would be interesting to link the drawing of an automaton with the set of functions in a code implementing this automaton. And when navigating through the software, it is crucial to be able to go from a given automaton state in the graphics to the code implementing it and vice versa.

And of course, one needs a system agnostic documentation software, running on many platforms and working with all possible kind of IDE and languages.

Overall, one needs an integrated documentation system that is able to encompass all the kind of artefacts that make a software and that can be used to maintain those artefacts along the whole life of the software, from design to maintenance. And this system should be flexible enough to not constrain you to a fixed development environment, letting you chose the tools that you prefer.

An interesting proposal in that regard is Scribble, a system for writing library documentation, users guides and tutorials. Scribble allows to write LaTeX like documents, JavaDoc like documents to document libraries or WEB-like literate programming documents. However Scribble is tightly linked to the Scheme language and thus lacks the important platform independence feature.

As you see, my proposed ideas are rather general and I am very far from being able to specify the documentation software I would like to have. I know for sure that the current documentation systems are not satisfying and I hope somebody will propose one day a better system, matching the needs we have to build better software. I would prefer not to have to implement it by myself. ;-)


[1] Knuth's literate programming system was called WEB, long before the Web was invented. :-)

Thu 01 Apr 2010

Pourquoi je quitte l'expérience démocratique

demexp On m'a récemment contacté à propos de l'expérience démocratique. J'avais déjà annoncé que je quittais le projet mais j'ai fini par élaborer une réponse plus précise. Il me semble intéressant de la rendre publique, non pas pour étaler mes états d'âmes mais comme une réflexion a posteriori sur un engagement et sur un projet de logiciel Libre qui n'a pas marché (et accessoirement pour l'« histoire » du projet ;-).

Si on retourne en 2003, je venais surtout pour l'aspect informatique du projet. Faire un logiciel de débat et de vote électronique original, c'était intéressant et très motivant. Mais après plusieurs années de développement je n'ai toujours pas un logiciel utilisable et je me suis retrouvé tout seul à le faire (ou quasiment). La lassitude a fait le reste du travail. ;-)

Par ailleurs, certains choix technologiques ont limité le développement du projet (client lourd en Gtk+, pas d'interface web), même si cela était dû à des contraintes quand j'ai commencé (Ocsigen n'existait pas à l'époque). Pour d'autres choix comme utiliser le langage OCaml, ça a très certainement limité l'arrivée de nouveaux développeurs. Mais je n'en aurais pas fait autant dans des langages plus classiques comme PHP ou Python. D'ailleurs d'autres ont proposé des morceaux de logiciel, en repartant de zéro comme Jean-Marc Fauché ou Lyu Abe. Mais ces propositions ne m'ont jamais emballé.

Plus sur le fond, je m'interroge pas mal sur le sens du projet. La démocratie ne se limite pas au vote, le débat et la formation des citoyens sont tout aussi important. Après quelques années, j'ai eu l'impression qu'on se concentrait uniquement sur l'aspect vote et pas assez sur les autres aspects. Pour le dire autrement, c'était peut-être plus important de faire un serveur de listes de diffusion, un service web pour synthétiser les débats et puis finalement voter en Condorcet avec du papier et du crayon que de faire un logiciel de vote électronique. On avait dit qu'on ferait des « soirées Condorcet » dans un bar, pour mettre en œuvre de manière concrète et ludique les idées du projet : cela ne s'est jamais fait.

Pour le logiciel, je suis parti sans réellement de spécifications, quasiment par le codage. Ce n'est pas comme ça que l'on fait un bon logiciel. ;-) Évidemment, quand on a démarré, on ne savait pas trop où on allait. Maintenant je pense que j'aurais des idées plus claires.

Informatiquement parlant, l'amplitude du projet est très vaste. Cela va des bases de données aux interfaces graphiques en passant par la cryptographie ou le réseau. C'est très stimulant sur le plan intellectuel, on apprend pleins de choses mais on n'est pas forcément bon partout.

Sur un plan plus pratique et citoyen, le vote électronique est très polémique (voir par exemple ordinateurs-de-vote.org). Je suis moi même contre les machines de votes actuelles[1] ! Il est quasiment impossible de faire un logiciel de vote aussi transparent qu'une urne en plexiglas. La recherche dans le domaine des protocoles de vote avance mais il faudra encore probablement de nombreuses années avant d'avoir quelque chose d'utilisable. Et ce sera très vraisemblablement plus compliqué qu'un bulletin dans une urne.

Sur le plan plus humain, on a démarré à quatre sur le projet, puis à trois au cœur. J'ai trouvé sur le long terme que j'étais un peu seul à faire la gestion de tâches informatiques (mise à jour du serveur, répondre sur les listes, le développement, etc.), malgré le soutien de mes deux compères. Ils ont beau être les pères de l'idée, j'ai trouvé que leur implication n'y était plus.

Enfin et surtout, après plusieurs années, j'ai envie de m'attaquer à d'autres défis, dans des domaines totalement différent et cela ne se fera pas sans dégager du temps libre. J'ai donc arrếté mon implication dans des projets non « productifs ».

Voilà, c'est probablement trop long et un peu confus, mais c'est dur de résumer plusieurs années de projet en quelques mots.

Je considère toujours que la thématique du projet, la réflexion sur la démocratie et le rôle du politique, est cruciale. Peut-être juste que l'angle n'était pas le bon, je ne sais pas. Apparemment, d'autres on des idées similaires et très intéressantes.

Et l'expérience a été pour moi enrichissante. Je ne regrette rien ! :-)


[1] Vous avez dit schizophrène ? ;-)

Mon 29 Mar 2010

Status of OCaml packages on Ubuntu Lucid Lynx (10.04 LTS): transition to OCaml 3.11.2 finished

I don't know who are responsible for this but the OCaml packages of Ubuntu Lucid Lynx 10.04 LTS have all transitioned to OCaml 3.11.2 on main architectures (amd64 and i386). A big thank to the mysterious developer(s)! Even for secondary architectures, all packages have transitioned to 3.11.2 except 3 packages on armel: coq, ssreflect and why.

Of course, having a source OCaml package compiled with the correct version of the OCaml compiler does not make it automatically working so I encourage you to test your preferred Ubuntu OCaml packages in Lucid.

If we now compare the set of source packages available respectively on Debian Unstable and Ubuntu Lucid, the situation is not so perfect. On the 145 OCaml packages available in Unstable, 21 are not at the same stage in Lucid.

There are 5 packages simply not available in Ubuntu:

  • clang 2.6-3
  • llvm 2.6-8
  • llvm-snapshot 20100312-1
  • obrowser 1.1+dfsg-4
  • unison2.27.57 2.27.57-2

There are 11 packages that have been updated in Unstable but not upgraded in Lucid:

  • Package Unstable-version Lucid-Version
  • approx 4.2-1 4.1-1
  • camlpdf 0.5-1 0.4-4
  • coccinelle 0.2.2.deb-1 0.2.0.deb-1ubuntu2
  • graphviz 2.26.3-3 2.20.2-8ubuntu3
  • ocaml-csv 1.2.0-1 1.1.7-2
  • ocaml-ssl 0.4.4-1 0.4.3-3
  • ocaml-text 0.3-1 0.2-3
  • ocsigen 1.3.0-4 1.2.2-1
  • pgocaml 1.4-1 1.3-3
  • postgresql-ocaml 1.12.4-1 1.12.1-2
  • unison 2.32.52-1 2.27.57-2ubuntu2

And lastly there are 5 packages that had minor updates or packaging bug fix in Unstable but not in Lucid:

  • Package Unstable-version Lucid-Version
  • nurpawiki 1.2.3-4 1.2.3-3
  • frama-c 20090902+beryllium+dfsg-5 20090902+beryllium+dfsg-4
  • ocamlgraph 1.3+debian-2 1.3+debian-1
  • sks 1.1.1-2 1.1.1-1ubuntu2
  • ssreflect 1.2+dfsg-4 1.2+dfsg-3

I don't know what to do about those packages or if I can even do anything. According to Ubuntu Lucid release schedule, we are reaching Beta 2 Freeze (on April the 1st) where uploads for packages in main and restricted are held in the queue and are subject to manual approval of the release team.

Do you have any advice?

Beside that, we still have 124 OCaml source packages in good shape in Lucid!

Looking for a C software for Formal Verification

As you probably know, I'm a huge fan of Formal Methods: use appropriate Mathematics and tools to ensure a program is correct in all possible situations. In other words, bug free software... well, sort of. :-)

The interesting side of this is that tools to apply Formal Methods have improved a lot and most of them are now Free Software. I'm maintaining a list of Free Software tools for Formal Methods (it is a wiki, you can update it!).

I would like to make an experiment with Frama-C and its plugins, especially Jessie. Frama-C is a framework for static analysis of C programs developed at CEA. Combined with the Why and Alt-Ergo tools, you can prove some properties on real C code (absence of integer underflow or overflow, absence of out-of-bound accesses, absence of NULL pointer de-referencing, program's specific properties, etc.). All those tools are Free Software and are developed in OCaml. And they now are available in Debian and Ubuntu!

I made a simple experiment last year but I would like to make a more elaborated one.

Therefore, I'm looking for a piece of C code with following criteria:

  • Free Software: I'm interested in improving software for the whole humankind; ;-)
  • Pure C code, no C++. If there is some assembly, I could work around for example by re-writting corresponding C function;
  • Code of moderate size, a few thousands line at most. It could be a sub-module or subset of a bigger code;
  • Code using mostly integers and pointers, few strings (aka char *)[1];
  • Verifying some properties on this code would be "interesting". Several possible reasons: for security or safety reasons, because the code is used in an embedded platform on which modifications are difficult once in production or simply because this code is used a lot.

If you know some software that fills those criteria, let me know through a comment or at dmentre@linux-france.org!


[1] Frama-C is a bit slow to handle strings and it can become cumbersome.

Thu 14 Jan 2010

Quick news: OCaml on Ubuntu Lucid and MapOSMatic

OCaml on Ubuntu Lucid

I have updated my scripts to compare Ubuntu OCaml packages to Debian ones. This time, I'm comparing Ubuntu Lucid against Debian testing, as for Lucid packages are imported from Debian testing (because Lucid is a Long Term Support release).

You'll find all the generated files here: http://bentobako.org/ubuntu-ocaml-status/raw/


As you have probably seen, we have done major improvements to MapOSMatic during Christmas, at both the web site level and the rendering level. I won't go into details, just read our initial announcement. Since then, we are continuing our improvements on maposmatic web front-end and ocitysmap back-end, with a new web site layout, translation of web site and maps in many languages (Arabic, Brazilian Portuguese, Catalan, Dutch, French, German, Italian and Russian). Many thanks to the numerous contributors!

We still have a lot of things to do or bugs to fix but the feedback is very positive and rewarding! Many thanks!

Thu 19 Nov 2009

OCaml on Ubuntu: looking for a new maintainer

HELP At some point I helped keeping the OCaml packages on Ubuntu in good shape, especially for the Karmic 9.10 release.

Unfortunately, I have much less free time those days and can no longer monitor OCaml packages on Ubuntu. Is anybody willing to work on this?

The main job is to look at the Debian packages and check if they are currently available in Ubuntu, and rebuild them if necessary. When the OCaml compiler changes (fortunately not so often), one needs to trigger a rebuild of all packages and that can be a bit difficult, mainly because LaunchPad does not provide an interface to rebuild several packages, taking into account their dependencies.

Of course, I would help anybody willing to do that job (explain the needed scripts, issues I had, etc.).

page 1 / 2