On Sun, Jul 21, 2013 at 6:54 AM, Stephan Beal <***@googlemail.com> wrote:
> This topic has been tossed around before, but the amount of effort
> involved in its undertaking has always kept us from actually doing it...
> To help bootstrap the process of figuring out what Fossil v2 might look
> like i have started writing down ideas in a public Google Doc:
> If you don't have access to that doc, either send me your gmail address
> and i'll gladly add you, or post your ideas here and i'll integrate them
> into the doc.
Some random thoughts on Fossil v2 as a library (call it "libfossil2") and
as a default client / server binary which makes use of the library (call it
"fossil2client" or "fossil2scm", I seem to have used both names), which I
realize you might already know about or have thought of but just in case...
(if only because I have a terminal case of Male Answer Syndrome) (and, wow,
this got longer than I expected, and I apologize in advance for telling you
in detail all about everything you Probably Already Knew and moreover Knew
Far More About Than I Do...):
* Fossil2client should *not* expect or require libfossil2 to be compiled
into it statically, or have an embedded copy of the source for libfossil2
within the source of fossil2client.
Reason why: The various Linux distributions, certainly the major FOSS ones
such as Fedora, Debian, etc, will require this; indeed, if fossil2client is
shipped with an embedded copy of the source for libfossil2, that embedded
copy will be stripped out in favor of the independent library version of
libfossil2 and fossil2client will be compiled to use that independent
library copy. They don't want to worry about security and other bugs they
have to patch in embedded code copies, etc; they want to fix the bug in one
place, the independent library itself, and let all the users of the library
just naturally benefit from that fix.
Debian, which is the Linux distro I am most familiar with, already does
this (or appears to anyway) with fossil v1, see
http://packages.debian.org/jessie/fossil where the fossil binary depends on
libsqlite3-0 (>= 3.7.11 at this instant). I don't know if fossil v1 ships
with an embedded code copy of sqlite (I haven't looked), but if it does
they're stripping it in favor of sqlite as an independent library object
and making their binary for fossil v1 dependent on that independent sqlite
I fully expect they would / will do the same with fossil2scm as a
dependency of libfossil2.
* The library should be implemented in such a way that multiple versions /
generations of it can be installed simultaneously. Perhaps alternatively,
there should be a mechanism where the library can be queried as to which
API version it is supporting, and/or instructed as to which API version a
client using it understands how to use.
Reason why: Let's say libfossil2 is installed and is supporting API version
Apple. Let's say two clients making use of libfossil2, namely fossil2scm
and IDEusingfossil2, are installed, and both know how to use libfossil2
with API version Apple.
Now, let's say a new release of libfossil2 is made with API version Berry
(Berry > Apple). Let's say there is some reason that libfossil2 with API
version Berry is not fully compatible with clients that only know how to
use API version Apple. Now, fossil2scm, because it comes from the fine
folks who brought you libfossil2, has a new version released in conjunction
with the new version of libfossil2 and the new version of fossil2scm is
compatible with API version Berry. However, IDEusingfossil2 does not yet
have a new release that understands API version Berry.
A Linux distribution faced with this, which wishes to offer working
versions of libfossil2, fossil2scm, and IDEusingfossil2, needs to be able
to allow for multiple versions of libfossil2 to be installed simultaneously
if it wishes to offer the very latest available versions of libfossil2 and
fossil2scm (because until IDEusingfossil2 can catch up in terms of
development it cannot use the very latest available version of
libfossil2). Alternatively, the Linux distribution is forced to not offer
the very latest available version of libfossil2 and fossil2scm until such
time as other things which depend on libfossil2 (e.g. IDEusingfossil2) can
be updated to use API version Barry, or else the distribution needs to make
things using API version Berry (e.g. fossil2scm) not simultaneously
installable with things using API version Apple (e.g. IDEusingfossil2).
However, note that if IDEusingfossil2 is able to tell whatever version of
libfossil2 it is linked with "I understand API version Apple, and I need
you to behave precisely as if you are API version Apple", and if libfossil2
is willing and able to obey this requirement, then it would be possible to
update libfossil2 (and other dependencies) to API version Berry even tho
IDEusingfossil2 is not yet updated to understand that API.
* The library should be implemented in such a way that multiple instances
of the library, and/or multiple different clients using either the same
instance of the library and/or multiple different instances of the library,
will not accidentally corrupt fossil repositories. Note that under
Unix/Linux many types of file locking (especially network file locking) are
less than fully secure or reliable.
* It might be worthwhile to reach out to the various major Linux and BSD
Unix distros to see what will make their lives easier in terms of packaging
and distributing libfossil2 and fossil2scm. This probably isn't as much of
an issue for people using Microsoft Windows and I assume MacOS/X, because
those people will mostly or solely use the binaries the fossil project
itself provides for libfossil2 and fossil2scm; only the most dedicated
developer will want to compile it themselves.
While I am reading the "Random Thoughts on Fossil v2" Google Doc circa the
time I am writing this:
Localization / internationalization: For at least FOSS projects, I know
there are organizations out there who will do much of the work of
localizing and internationalizing the software for you (in particular
various language strings) As Long As you use software libraries which make
it easy for them to do this. Whether or not these software libraries would
affect libfossil2 and fossil2scm in terms of the software licenses
applicable to them (I'm thinking in particular of GPL / strong copyleft
licensing as opposed to LGPL / weak copyleft or MIT/BSD/Apache/etc
non-copyleft licensing), I do not know off the top of my head. But, the
possibility is out there at least to leverage the work these groups do.
Abstraction around the repo and config DBs: I see two main areas here --
(1) abstractions which maintain the use of a SQL database, but don't
necessarily mandate sqlite, e.g. MySQL (and its siblings) or PostgreSQL, or
on Windows Microsoft SQL Server, or on everything Oracle, could possibly be
used as the database engine; (2) abstractions which do not mandate the use
of SQL, and in particular ones which permit use of some flavor of a
NoSQL-type engine. (I note that there are several distinct and not
necessarily cooperative things which all fall under the banner of NoSQL.)
Whether or not any of this would be desirable or even worthwhile I
certainly have no idea.
On a semi-related note... I know (or am under the impression) that many
database engines like to try to get down to the "raw disk" / bare metal,
and not have the platform OS be an intermediary, in terms of their access
to their files. I don't know if or how this is applicable to libfossil2
but I thought I would mention it.
On another note... We have SSDs now which are very fast but have limited
write cycles. We also have RAM disks available on many platforms. Does
this have any significance to libfossil2 and/or fossil2scm?
OS-level peculiarities: There are other multi-platform software projects
which have to deal with these, both with peculiarities between similar
types of platform, such as the various flavors of Unix and Linux, and
between drastically different platforms, such as between the former versus
the different versions of Windows, MacOS/X, ummm, VMS, the IBM mainframe
OSen, etc etc etc. All the major FOSS SQL databases have to deal with
this, the Apache web server (Apache HTTPD) has to deal with this (they have
Apache Portable Runtime IIRC), Gnu Emacs has to do this (tho that would
certainly involve GPL licensed software), I'm sure there's plenty more...
Maybe there's stuff that can be stolen ^H^H borrowed from one or more of
those to avoid having to roll your own.
Strict C89 or commonly-supported C99 extensions: What platforms, and
compilers on those platforms, do you want to be portable to? If everything
you care about is (non-buggy) C99, well...
HTTP/CGI servers: I could see people wanting to use libfossil2 with HTTP
servers other than what is provided by fossil2scm, e.g. Apache, Nginx,
Microsoft Windows IIS, whatever MacOS/X uses, etc.
Consistent CLI args/flags handling: For Linux, the Gnu C Library (libc) has
getopt and argp_parse (see
details). However, this is certainly not available as a standard
library for platforms like Microsoft Windows and VMS, and if it is provided
as a library which is not a standard system library there are likely GPL
licensing issues. It might be worth seeing what the projects I mentioned
in OS-level peculiarities do about this, and what the various BSD Unix
flavors do about this.
On a semi-related note, and because I'd written most of it before realizing
this wasn't applicable to CLI args/flags... If fossil2scm is to be used in
an interactive mode (e.g. where it does not fire up, do something, and
terminate, but instead provides some sort of internal command line or
windowing interface as applicable)... Certainly there's Very Well Known
libraries, at least for Unix/Linux, which Just Handle This, e.g. Gnu
Readline. However, Gnu Readline is GPL licensed (and will *NOT* change),
so unless this license is acceptable for libfossil2 / fossil2scm a
different library with a more acceptable license would be required. (I do
not know what the BSD Unix equivalent to Gnu Readline is, but I think one
is being worked on with a license more acceptable to the BSDs.)
Not restricting to backwards compatibility: If you do this, libfossil2 and
things dependent on it must never write something to a fossil v1 repository
which would be seen by a v1 client as corruption, unless, *perhaps*, there
is an Explicit and Unambiguous direction to do so with acknowledgement that
this will render the v1 repository corrupt in the eyes of a v1 client.
This requires that libfossil2 always be able to correctly determine that a
repository is or is not in v1 format (or perhaps more safely assume that a
repository is always in v1 format unless they can specifically determine
that it is v2 format).
Social networking logins: See the OpenID Foundation (at http://openid.net/).
On a related note... For SSL, PGP keys, etc... I know there's issues with
clients being able to tell that the server they are being directed to by
means of DNS is the real server they want to go to, e.g. that they are not
being fed poisoned or spoofed DNS data. IIRC there are protocols being
developed for DNS to deal with this sort of issue, tho I don't know how
mature they are yet or how widely they're implemented. For (public) PGP
keys, see the stuff at www.pgp.net (it redirects). Also related
http://en.wikipedia.org/wiki/CAcert and http://www.cacert.org/ .
On a different note... Insofar as networking is an issue, make certain
everything works with IPv6 as well as IPv4!
Goodness. I didn't realize I would write that much.
In any event. Thanks for giving me a (large) chunk of your time by reading
this. Hope at least some of it has been of some use, interest. Be well.