[fossil-users] Fossil with IPv6 support on Windows XP

[Olivier Mascia]

Dear,

I could finally re-install a VM with a plain old stock "Windows XP with Service Pack 3", to re-run a series of tests, because I had some fears. Here are some findings for you to consider.

Out of fresh install, IPv6 is not setup. So fossil ui and fossil server will both fail listening on a socket. That is expected. Actions where fossil acts as a client (fossil clone/push/pull for instance) will work correctly though.

On Windows XP, you have to run the command "ipv6 install" to add the protocol to the stack (much simpler than through the GUI). A reboot is NOT required (as incredible as it looks).

After that, again, fossil as a client (fossil clone/push/pull) will work correctly this time either to an IPv4 or IPv6 target.

Though, fossil ui will fail with "unable to open listening socket", while fossil server will succeed.

The root culprit is here: https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/sag_ip_v6_imp_config_items.mspx?mfr=true

in the "Special addresses" paragraph:
"The IPv6 protocol for Windows (XP) does not support the use of IPv4-mapped addresses."

As implementing a dual-stack listening socket implies IPv4-mapped addresses, this explains the problem. It could be solved by modifying fossil server/ui to listen on two distinct sockets (one pure IPv4 and one pure IPv6). That would also work on all other Windows versions.

I think this is not a very useful complication. Anyone still requiring to use fossil on such archeologic thing as Windows XP won't have any specific issue with fossil, except the requirement to run fossil server instead of fossil ui before accessing the content through http://localhost (that will work nicely). It just kills the prospect of setting up a fossil server (on Windows XP only) for generic access from both IPv4 and IPv6 clients. I suppose that is a small limitation people can live with, seeing, if I'm not mistaken, that fossil doesn't yet support IPv6 at all on unixes.

[Florian Balmer]

Re: Fossil with IPv6 support on Windows XP

I have several physical and virtual XP machines around, and `fossil
ui' doesn't work any more with the new patch.

--Florian

[Marcelo Huerta]

Florian Balmer decía, en el mensaje "Re: [fossil-users] Fossil with IPv6

Post by Florian Balmer
Re: Fossil with IPv6 support on Windows XP
I have several physical and virtual XP machines around, and `fossil
ui' doesn't work any more with the new patch.

I can add that starting with 21d5038f, for "fossil server", localhost access
without login stops working even when the --localauth is passed on invocation,
at least in my Windows 7 installation.

I use this to access all my local repos without login so I reverted to 6b5cfc86.

[Richard Hipp]

Post by Marcelo Huerta
I can add that starting with 21d5038f, for "fossil server", localhost access
without login stops working even when the --localauth is passed on invocation,
at least in my Windows 7 installation.

I think this should be fixed on the latest trunk check-in
(https://www.fossil-scm.org/fossil/info/96dcb7e709a47863) Please try
it out and let me know either way.

On the other hand, I don't know how to fix Florians WinXP problem, nor
do I have access to a WinXP machine for testing. Maybe Olivier can
help with that....

[Richard Hipp]

Post by Richard Hipp
I think this should be fixed on the latest trunk check-in
(https://www.fossil-scm.org/fossil/info/96dcb7e709a47863) Please try
it out and let me know either way.

Sadly, it still asks for a login for localhost.

Can you try again on using
https://www.fossil-scm.org/fossil/info/c038de8d27b5ef14 please, and
let me know. Thanks for your help.

[Florian Balmer]

Thanks for the interesting details about IPv6 and Windows.

Post by Richard Hipp
I'd appreciate it if some readers could try the two versions out,
side by side, and give their opinions.

Focusing on Windows XP here, the version with [c038de8d] works fine
and navigates to the correct page for `fossil ui'.

Hard to say anything reasonable about speed on Windows XP, as Google
Chrome version 49.0.2623.110 is the latest XP-available browser to
handle all the JavaScript and timeline drawings correctly.

The startup delay for `fossil ui' and `fossil server' on Windows XP is
more obvious than possibly sluggish browser navigation, which I
*think* is due to waiting for StartServiceCtrlDispatcherW. This could
be cut down by skipping the call to StartServiceCtrlDispatcherW for
the `ui' and `server' commands, as Fossil always runs in a console
session, and not as a service, in these cases.

--Florian

[Thomas Schnurrenberger]

Post by Florian Balmer
The startup delay for `fossil ui' and `fossil server' on Windows XP is
more obvious than possibly sluggish browser navigation, which I
*think* is due to waiting for StartServiceCtrlDispatcherW. This could
be cut down by skipping the call to StartServiceCtrlDispatcherW for
the `ui' and `server' commands, as Fossil always runs in a console
session, and not as a service, in these cases.

I measured the time it takes to call StartServiceCtrlDispatcherW on my
8 year old Windows 7 64bit box: roughly 600 microseconds, I don't think
this is noticeable!

This is the program for taking the time:

***** Start of code *****
#include <stdlib.h>
#include <stdio.h>
#include <inttypes.h>
#include <windows.h>

static void WINAPI win32_http_service_main(
DWORD argc,
LPWSTR *argv
){
return;
}

int main(void){
LARGE_INTEGER Frequency;
LARGE_INTEGER StartingTime;
LARGE_INTEGER EndingTime;
LARGE_INTEGER ElapsedMicroseconds;
double ElapsedSeconds;
int rc = 0;

/* Define the service table. */
SERVICE_TABLE_ENTRYW ServiceTable[] =
{{L"", (LPSERVICE_MAIN_FUNCTIONW)win32_http_service_main},
{NULL, NULL}};

/* Get frequency and the start time. */
QueryPerformanceFrequency(&Frequency);
QueryPerformanceCounter(&StartingTime);

/* Activity to be timed */
if( !StartServiceCtrlDispatcherW(ServiceTable) ){
if( GetLastError()==ERROR_FAILED_SERVICE_CONTROLLER_CONNECT ){
rc = 1;
}else{
rc = 2;
}
}
/* Get end time and convert to seconds. */
QueryPerformanceCounter(&EndingTime);
ElapsedMicroseconds.QuadPart = EndingTime.QuadPart
- StartingTime.QuadPart;
ElapsedMicroseconds.QuadPart *= 1000000;
ElapsedMicroseconds.QuadPart /= Frequency.QuadPart;
ElapsedSeconds = (double)(EndingTime.QuadPart
- StartingTime.QuadPart)/Frequency.QuadPart;
printf("Elapsed microseconds: %"PRId64"\n",
ElapsedMicroseconds.QuadPart);
printf("Elapsed seconds : %f\n", ElapsedSeconds);
printf("rc : %d\n", rc);

return 0;
}
***** End of code *****

It would be interesting to known time on your XP boxes.

[i***@papier.host4free.ovh]

cool!

Post by Thomas Schnurrenberger

Post by Florian Balmer
The startup delay for `fossil ui' and `fossil server' on Windows XP is
more obvious than possibly sluggish browser navigation, which I
*think* is due to waiting for StartServiceCtrlDispatcherW. This could
be cut down by skipping the call to StartServiceCtrlDispatcherW for
the `ui' and `server' commands, as Fossil always runs in a console
session, and not as a service, in these cases.

I measured the time it takes to call StartServiceCtrlDispatcherW on my
8 year old Windows 7 64bit box: roughly 600 microseconds, I don't think
this is noticeable!
***** Start of code *****
#include <stdlib.h>
#include <stdio.h>
#include <inttypes.h>
#include <windows.h>
static void WINAPI win32_http_service_main(
DWORD argc,
LPWSTR *argv
){
return;
}
int main(void){
LARGE_INTEGER Frequency;
LARGE_INTEGER StartingTime;
LARGE_INTEGER EndingTime;
LARGE_INTEGER ElapsedMicroseconds;
double ElapsedSeconds;
int rc = 0;
/* Define the service table. */
SERVICE_TABLE_ENTRYW ServiceTable[] =
{{L"", (LPSERVICE_MAIN_FUNCTIONW)win32_http_service_main},
{NULL, NULL}};
/* Get frequency and the start time. */
QueryPerformanceFrequency(&Frequency);
QueryPerformanceCounter(&StartingTime);
/* Activity to be timed */
if( !StartServiceCtrlDispatcherW(ServiceTable) ){
if( GetLastError()==ERROR_FAILED_SERVICE_CONTROLLER_CONNECT ){
rc = 1;
}else{
rc = 2;
}
}
/* Get end time and convert to seconds. */
QueryPerformanceCounter(&EndingTime);
ElapsedMicroseconds.QuadPart = EndingTime.QuadPart
- StartingTime.QuadPart;
ElapsedMicroseconds.QuadPart *= 1000000;
ElapsedMicroseconds.QuadPart /= Frequency.QuadPart;
ElapsedSeconds = (double)(EndingTime.QuadPart
- StartingTime.QuadPart)/Frequency.QuadPart;
printf("Elapsed microseconds: %"PRId64"\n",
ElapsedMicroseconds.QuadPart);
printf("Elapsed seconds : %f\n", ElapsedSeconds);
printf("rc : %d\n", rc);
return 0;
}
***** End of code *****
It would be interesting to known time on your XP boxes.

[Olivier Mascia]

Post by Thomas Schnurrenberger

Post by Florian Balmer
The startup delay for `fossil ui' and `fossil server' on Windows XP is
more obvious than possibly sluggish browser navigation, which I
*think* is due to waiting for StartServiceCtrlDispatcherW. This could
be cut down by skipping the call to StartServiceCtrlDispatcherW for
the `ui' and `server' commands, as Fossil always runs in a console
session, and not as a service, in these cases.

I measured the time it takes to call StartServiceCtrlDispatcherW on my
8 year old Windows 7 64bit box: roughly 600 microseconds, I don't think
this is noticeable!

But on XP the delay is 'intolerable' rather than unnoticeable. :)

A small patch for that is included along a larger IPv4/IPv6 patch I submitted today.
I see Richard put it on trunk (see [e506ebb764]) minutes ago.

Especially those of you using Windows XP would help by building from that code and see how it goes for them.
It looks good, and does not require anymore to install IPv6 support (command 'ipv6 install') on Windows XP. The solution is IPv4/IPv6 agnostic. If at least one of both protocols are useable, the software should work nicely.

As I'm responsible for asking for this support and did a fair part of that coding, I'll stay alert on issues that would come out, in order to quickly help, should it happen.

[Florian Balmer]

Post by Thomas Schnurrenberger
I measured the time it takes to call StartServiceCtrlDispatcherW
on my 8 year old Windows 7 64bit box: roughly 600 microseconds, I
don't think this is noticeable!
It would be interesting to known time on your XP boxes.

Thanks for your interest and for the test program!

My results are:

Elapsed microseconds: 15000606
Elapsed seconds : 15.000607
rc : 1

This is for both my standard and tweaked physical machines mentioned
earlier [0], as well as a newly setup virtual box. Note that all my
systems are 32-bit.

[0] https://www.mail-archive.com/fossil-***@lists.fossil-scm.org/msg25001.html

A virtual box (32-bit) to reproduce the behavior can be setup quickly:

* Download "Windows XP Mode" [1]
* The downloaded file is an SFX archive containing a VHD
* Start the OS from the VHD with any virtualization tool

[1] https://www.microsoft.com/en-us/download/details.aspx?id=8002

I'm (still) aware that this is not an important problem, but I find it
interesting. I have some old development tools only available on my XP
machines (older versions of Visual Studio still providing Resource
Editors, to name it). Given the portability of Fossil, I was happy it
worked on Windows XP, but waiting 15 seconds for `fossil ui' cools
down my joy a little bit ;-)

--Florian

[Olivier Mascia]

Post by Florian Balmer
Given the portability of Fossil, I was happy it
worked on Windows XP, but waiting 15 seconds for `fossil ui' cools
down my joy a little bit ;-)

You might then enjoy a fresh build out of trunk.
:)

[Florian Balmer]

Post by Olivier Mascia
But on XP the delay is 'intolerable' rather than unnoticeable. :)
A small patch for that is included along a larger IPv4/IPv6 patch
I submitted today.
I see Richard put it on trunk (see [e506ebb764]) minutes ago.
Especially those of you using Windows XP would help by building
from that code and see how it goes for them.
It looks good, and does not require anymore to install IPv6
support (command 'ipv6 install') on Windows XP. The solution is
IPv4/IPv6 agnostic. If at least one of both protocols are useable,
the software should work nicely.
As I'm responsible for asking for this support and did a fair part
of that coding, I'll stay alert on issues that would come out, in
order to quickly help, should it happen.
You might then enjoy a fresh build out of trunk.

Goes off like a rocket, thanks for the nice work!

Also, using `GetStdHandle(STD_INPUT_HANDLE)!=NULL' to test whether or
not running in a console session is clever, my proposed solution was
to check for the `ui' and `server' commands.

--Florian

[Olivier Mascia]

Post by Florian Balmer

Post by Olivier Mascia
You might then enjoy a fresh build out of trunk.

Goes off like a rocket, thanks for the nice work!
Also, using `GetStdHandle(STD_INPUT_HANDLE)!=NULL' to test whether or
not running in a console session is clever, my proposed solution was
to check for the `ui' and `server' commands.

Glad you like it.
-You- quickly identified the culprit (StartServiceCtrlDispatcherW out of the context of a Service run), which at least on XP has a lengthly side-effect.
:)

[Richie Adler]

Florian Balmer decía, en el mensaje "Re: [fossil-users] Fossil with IPv6

Post by Florian Balmer
Re: Fossil with IPv6 support on Windows XP
I have several physical and virtual XP machines around, and `fossil
ui' doesn't work any more with the new patch.

I can add that starting with 21d5038f, for "fossil server", localhost access
without login stops working even when the --localauth is passed on invocation,
at least in my Windows 7 installation.

I use this to access all my local repos without login so I reverted to 6b5cfc86.

[Olivier Mascia]

Post by Florian Balmer
Re: Fossil with IPv6 support on Windows XP
I have several physical and virtual XP machines around, and `fossil
ui' doesn't work any more with the new patch.
I can add that starting with 21d5038f, for "fossil server", localhost access
without login stops working even when the --localauth is passed on invocation,
at least in my Windows 7 installation.
I use this to access all my local repos without login so I reverted to 6b5cfc86.

Which I offered for discussion in my post "Fossil with IPv6 support on Windows XP" yesterday.

## Additional fix (seeing today's reports)

You will have to do two things.

First, from a command prompt on Windows XP (>=SP1), type:

ipv6 install

And second, use code from or above [723dedac] fossil's trunk *and* apply this additional patch:

Index: src/winhttp.c
==================================================================
--- src/winhttp.c
+++ src/winhttp.c
@@ -376,11 +376,11 @@
#endif
if( WSAStartup(MAKEWORD(2,0), &wd) ){
fossil_fatal("unable to initialize winsock");
}
if( flags & HTTP_SERVER_LOCALHOST ){
- zIpAddr = "127.0.0.1";
+ zIpAddr = "::1";
}
while( iPort<=mxPort ){
DWORD ipv6only = 0;
s = socket(AF_INET6, SOCK_STREAM, 0);
if( s==INVALID_SOCKET ){


Merely replacing "127.0.0.1" with "::1".

## Why?

On modern Windows versions, IPv6 is configured by default and often preferred by the system when attempting outgoing connections as soon as DNS reports both AAAA and A records. When enumerating IPs associated with a name, Fossil tries each IP in sequence until it finds one which connects. If the fossil server you try to reach has IPv6 addresses published in DNS, chances are those will be tried first, even though there are IPv4 addresses also listed. This leads to *long* pauses (about 10 to 20 seconds for each such try) when Fossil server is based on code *before* [76c7a9a7], just because fossil server/ui simply did not accepted IPv6 connections, but only IPv4 connections.

This is why IPv6 support has been added to fossil server/ui on Windows. That is a *good* thing anyway because if the OS support or can support IPv6 it shouldn't be ignored by any application.

To implement IPv6 server/ui support there are two paths. Either listen on two sockets: one IPv4 *and* one IPv6. Or listen to a single dual-stack socket. That path was chosen because this is the usual proper way to do it, at least on Windows, but I'm rather confident, on unixes too. Also this is the path of least structural change to existing code.

A dual-stack socket on Windows is primarily an IPv6 socket which can also accept IPv4 traffic. To this effect, IPv6 (the standard, not a Windowsish thing this time) do provide the special range of IPv6 addresses ::FFFF:w.x.y.z to represent IPv4 addresses mapped to IPv6 address-space (we're not talking tunneling of any sort here). That is just a reserved range, just for that purpose, within the enormous IPv6 address-space.

There is a caveat (on Windows XP).

On Windows XP, IPv6 is not pre-installed. You need at least SP1 to have support for IPv6, but it isn't anyway configured by default. There is a convenient system utility named "ipv6" on Windows XP that serves as the way to configure IPv6 settings which aren't accessible through any GUI component.

So first thing first, you need to add IPv6 support by running the command-line:

ipv6 install

Takes 5 seconds, and as incredible as it may be, doesn't require a reboot.

But there is one additional caveat, which the above patch treats.

Windows XP does support IPv6 (as above) and IPv4, but falls short supporting dual-stack sockets because it doesn't recognise or support in any way those IPv4 addresses mapped to IPv6. So ::ffff:127.0.0.1 which is the way to define IPv4 127.0.0.1 in IPv6 way is not understood at all by the system.

The above minimalist patch does instruct Windows to listen on the pure IPv6 "localhost" which is ::1, instead of listening on the pure IPv4 "localhost" which is ::ffff:127.0.0.1 in IPv6 notation.

## What's the effect?

On Windows XP, "fossil ui" will work again. It will listen to ::1 (IPv6) and launch a browser to connect to "http://localhost", which will get mapped to "::1" and will succeed.

On other windows, the situation is unchanged.

## Other paths

A probably better way of implementing fossil ui on Windows XP would be to actually listen on two sockets (IPv4 and IPv6 separately). But that is NOT needed on other Windows versions (later than Windows XP at least).

I think that with the added above patch, things should get back to normal for Windows XP users, provided they agree to run once and for all "ipv6 install" to add IPv6 protocol support to their system.

Open for discussion, of course, but please first test with [723dedac] along the above one line change patch.

On my Windows XP test VM which happen to be XP Professional SP3, it works and looks just fine (provided you have installed some other browser than the stock IE of course).

Depending on return from people, I will happily spend more time offering our community a more complex / complete solution, that wouldn't impact in any way Windows XP users, including retiring the need to add IPv6 protocol. Though I think that is probably not needed.

[Richard Hipp]

Post by Olivier Mascia
And second, use code from or above [723dedac] fossil's trunk *and* apply
if( flags & HTTP_SERVER_LOCALHOST ){
- zIpAddr = "127.0.0.1";
+ zIpAddr = "::1";
}

This patch, plus a couple of others are now on a branch:
https://www.fossil-scm.org/fossil/info/c038de8d27b5ef14

It works on both Win10 and Win7. But it seems slower for "fossil ui"
than using IPv4. I changed the URL to be "http://[::1]:8080/" instead
of "http://localhost:8080" and that seemed to help it some. But it
still seems more sluggish.

But perhaps this is just my imagination.

I'd appreciate it if some readers could try the two versions out, side
by side, and give their opinions.

[Richie Adler]

Post by Olivier Mascia
Merely replacing "127.0.0.1" with "::1".

In my Windows 7 x64 installation, updating to 96dcb7e7 and making the
indicated change DOES NOT restore the --localauth functionality.

[i***@papier.host4free.ovh]

Hallöchen!


Von meinem iPhone gesendet