Information for developpers
OVERVIEW
Although the sources of camlp5 are written in camlp5 syntaxes, the
distribution contain no 'magic' binary file for the first compilation.
This first compilation can be done thanks to a version of the core of
camlp5 written in pure ocaml syntax. Once this 'core' is compiled, all
sources in revised syntax with syntax extensions can be compiled.
COMPATIBILITY
This distribution of camlp5 is compatible with several versions of
ocaml. The problem is about the definition of ocaml syntax trees which
may change from a version of ocaml to another. Since ocaml does not
install the sources nor the compiled versions of its syntax tree, a
copy of the necessary source files, borrowed from the source of the
ocaml compiler has been done in the directory 'ocaml_stuff', in
subdirectories with the ocaml version number.
If the present distribution of camlp5 is not compatible with the
version of ocaml you have (the command 'configure' tells you), it
is possible to add it. For that, you need the sources of the ocaml
distribution you have. Once done, after a 'configure' telling you
that it is not compatible, do:
make steal OCAML_SRC=<path-to-ocaml-sources>
This creates a new directory in 'ocaml_stuff' with sources of the
syntax tree of your ocaml compiler.
If you want to check that the sources of the syntax tree of ocaml
are up-to-date (e.g. if this is the current ocaml developpement), do:
make compare_stolen OCAML_SRC=<path-to-ocaml-sources>
The compatibility is also done with the file 'lib/versdep.ml',
which is a module containing miscellaneous features depending to
the version of OCaml.
In the directory 'ocaml_src' which contains the pure OCaml sources
of the Camlp5 core (see chapter TREE STRUCTURE below), there are as
many versions of this files as versions of OCaml. They are named
'<version>.ml' in the directory 'lib/versdep'. If you are adding
a new version of OCaml, you need this file. As a first step, make a
copy from a close version:
cd ocaml_src/lib/versdep
cp <close_version>.ml <version>.ml
Then, you can redo "configure" and do "make core". If the file
'ocaml_src/lib/versdep.ml' has a compilation problems, fix them and
to 'make core' again.
Later, the same file 'lib/versdep.ml' in camlp5 syntax may have
similar compilation problem. This file is in one examplary, thanks to
IFDEF used here or there.
While compiling with some specific version of ocaml, this file is
compiled with 'OCAML_vers' defined where 'vers' is the version number
form the beginning to the first space or charcter '+' with all dots
converted into underscores. For example, if your ocaml version is
7.04.2+dev35, you can see in the compilation process of versdep.ml
that OCAML_7_04_2 is defined, and you can add statements defined
by the syntax extension 'pa_macro.cmo', for example IFDEF OCAML_7_04_2.
Add statements like that in 'lib/versdep.ml' to make it compile.
TREE STRUCTURE
The directory 'ocaml_src' contains images in pure ocaml syntax of the
directories odyl lib main and meta. This allow to create a core version
of camlp5 from only the ocaml compiler installed.
You can decompose the building of the camlp5 core into:
1. make library_cold
just makes the directory 'ocaml_src/lib' and copy the cmo and cmi
files into the directory 'boot'
2. make compile_cold
makes the other directories of ocaml_src
3. make promote_cold
copies the executables "camlp5", "camlp5r" and the syntax
extensions (cmo files) into the directory 'boot'
From that point, the core camlp5 is in directory 'boot'. The real
sources in the top directories odyl lib main and meta, which are
written in revised syntax with some syntax extensions (grammars,
quotations) can be compiled. To achieve their compilation, you can
do:
make core
Or to compile everything do:
make all
or just:
make
Notice that doing "make core" or "make all" from scratch (after a
make clean), automatically start by making the core files from their
pure ocaml versions.
FAST COMPILATION FROM SCRATCH
./configure
make clean core compare
make coreboot
make all opt opt.opt
TESTING CHANGES
1. do your changes
2. do:
make core compare
if it says that the bootstrap is ok, you can do:
make all
make opt
make opt.opt
otherwise, to make sure everything is ok, first do:
make coreboot
sometimes two bootstraps ('make coreboot' twice) are necessary,
in particular if you change things in the directory 'lib'. It is
even possible that three bootstraps are necessary.
If things goes wrong, it is possible to return to the previous
version by typing:
make restore clean_hot
then you can change what is necessary and continue by typing:
make core
and test the bootstrap again:
make coreboot
After several bootstraps (by 'make coreboot' or 'make bootstrap'),
many versions are pushed in the directory 'boot' (you can type
'find boot -type d -print' to see that). If your system correctly
bootstraps, you can clean that by typing:
make cleanboot
which keeps only two versions. (The command 'make clean' also removes
these stack of versions.)
BEFORE COMMITTING YOUR CHANGES
Make sure that the cold start with pure ocaml sources work. For that, do:
make compare_sources | less
This shows you the changes that would be done in the ocaml pure sources
of the directory ocaml_src.
To make the new versions, do:
make new_sources
make promote_sources
Notice that these pure ocaml sources are not supposed to be modified
by hand, but only created by the above commands. Besides, their sources,
although pretty printed, are sometimes not easy to read, particularly for
expanded grammars (of the statement 'EXTEND').
However, if these sources do not compile, due to changes in the ocaml
compiler, it is possible to edit them. In this case, similar changes
may have to be done in the normal sources in revised syntax.
After doing 'make new_sources' above, and before doing 'make
promote_sources' below, it is possible to do 'make untouch_sources'
which change the dates of the new created files with the dates of the
old files if they are not modified. This way, the "svn commit" will not
have to compare these files, which may have some importance if you
network is not fast.
The 'make new_sources' builds a directory named 'ocaml_src.new'.
If this directory still exists, due to a previous 'make new_sources',
the command fails. In this case, just delete it (rm -rf ocaml_src.new)
without problem: this directory is not part of the distribution, it is
just temporary.
The 'make clean_sources' deletes old versions of ocaml_src, keeping
only the last and the before last ones.
The command:
make bootstrap_sources
is a shortcut for:
make new_sources
make untouch_sources
make promote_sources
make clean_sources
If there are changes in the specific file 'lib/versdep.ml', do
also:
make compare_all_versdep
and possibly:
make bootstrap_all_versdep
because this file, in 'ocaml_src/lib/versdep' directory has different
versions according to the ocaml version.
After having rebuilt the pure ocaml sources, check they work by
rebuilding everything from scratch, starting with "configure".
IF YOU CHANGE THE MAIN PARSER
If you change the main parser 'meta/pa_r.ml', you may check that the
quotations expanders of syntax tree 'meta/q_MLast.ml' match the new
version. For that, do:
cd meta
make compare_q_MLast
If no differences are displayed, it means that 'q_MLast.ml' is ok,
relatively to 'pa_r.ml'.
Otherwise, if the displayed differences seem reasonable, update the
version by typing:
make bootstrap_q_MLast
Then returning to the top directory, do 'make core compare' and
possibly 'make coreboot' (one of several times) to check the
correctness of the file.
And don't forget, if you want to commit, to re-create the pure ocaml
sources like indicated above.
