Opened 3 years ago

Last modified 3 years ago

#25805 new task

Provide cygwin docker images for Windows

Reported by: saraedum Owned by:
Priority: minor Milestone: sage-wishlist
Component: distribution Keywords: docker
Cc: embray, slelievre Merged in:
Authors: Julian Rüth Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #24655 Stopgaps:

Status badges

Description

I have been playing with Windows (!) containers for docker on Windows recently. I would like to see whether it's feasible to provide a Dockerfile for such an environment similar to what we have in #24655.

For the end user this would probably not be very useful. I believe that one would need at least Windows 2k16 Server (probably the Standard Edition) to run these containers. The main use case would probably be in the CI context.

So far my impression is that native Windows containers are not as nice to work with as their Linux counterparts. They are really heavyweight and take quite a while to start up.

Change History (16)

comment:1 Changed 3 years ago by embray

Not having had access to such a server I've not yet even played with Windows native containers yet. I still need to get an Azure account to try that out, as I think it will also work better for the Windows patchbot...

comment:2 Changed 3 years ago by saraedum

I found (docker on) azure to be completely useless. It takes much longer than I am willing to wait to launch VMs there and I never got docker to work, apparently due to disabled virtualization options on the virtual machines.

I am now running Win2k16 Server Standard on QEMU and after some tweaking I am quite happy with the performance.

comment:3 follow-up: Changed 3 years ago by embray

I had just assumed that Microsoft would have put effort into making that work well, given the resources they've put into helping Docker implement it. If it's useless then why would the push it at all?

How did you even get this Windows Server, or is it unlicensed?

comment:4 follow-up: Changed 3 years ago by embray

But the question is, does Cygwin perform well? I have still never been able to get good Cygwin performance in a Windows VM on a Linux host (due mostly to fork, or at least that's my suspicion).

comment:5 in reply to: ↑ 3 ; follow-up: Changed 3 years ago by saraedum

Replying to embray:

I had just assumed that Microsoft would have put effort into making that work well, given the resources they've put into helping Docker implement it.

Azure is imho only nice to work with when using Linux VMs (maybe dedicated Windows hosts are also good, I have not tried.) They did not support docker on Azure at all when I tried a few months ago. I found this quite frustrating as I had assumed that everything should work out of the box there. Curiously enough, they recommend Linux/Docker? in some .NET Core tutorials for Azure…

If it's useless then why would they push it at all?

I have no clue.

How did you even get this Windows Server, or is it unlicensed?

It's a 180 day trial.

comment:6 in reply to: ↑ 4 Changed 3 years ago by saraedum

Replying to embray:

But the question is, does Cygwin perform well? I have still never been able to get good Cygwin performance in a Windows VM on a Linux host (due mostly to fork, or at least that's my suspicion).

The idea would be to run this with docker on a real Windows host eventually. I have not gotten far enough into the build process to make any statement about performance. If you have an easy benchmark (that does not require a built Sage) I am happy to try it out.

comment:7 in reply to: ↑ 5 ; follow-up: Changed 3 years ago by embray

Replying to saraedum:

Replying to embray:

How did you even get this Windows Server, or is it unlicensed?

It's a 180 day trial.

What happens after the trial period runs out?

comment:8 in reply to: ↑ 7 Changed 3 years ago by saraedum

Replying to embray:

Replying to saraedum:

Replying to embray:

How did you even get this Windows Server, or is it unlicensed?

It's a 180 day trial.

What happens after the trial period runs out?

I am not sure. I could move my containers to another trial I guess. I had a look at the license a few months ago. I don't remember the details, but I took away that I could play with it for as long as I wanted but could not use it for actual testing in production.

comment:9 Changed 3 years ago by saraedum

I did some benchmarks of forks with the following forkit.c. I ran the program on my Laptop (with a early 2015 i5) natively in Linux, on Windows 2k16 Server running on QEMU on Linux, in windowsservercore running on docker on Windows 2k16 Server running on QEMU on Linux. I also ran this on a native Windows 2k12 that runs on a somewhat aged Xeon processor for comparison.

invocation native Linux 2k16/QEMU docker/2k16/QEMU native 2k12 (different hardware)
./forkit 1024 1024 .8s 13s 50s 6s
./forkit 1024 10240000 1.2s 50s 80s 41s
#include <unistd.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <stdio.h>
#include <assert.h>

int main(int argc, char**argv){
  int F=atoi(argv[1]);
  int RAM=atoi(argv[2]);
  int* A=malloc(sizeof(int)*RAM);
  assert(A);
  for (int f=0;f<F;f++){
    int pid=fork();
    if (pid){
      printf("%d\n", f);
      int w;
      wait(&w);
    }else{
      for (int j=0;j<128;j++){
        A[rand()%RAM] = pid;
      }
      break;
    }
  }
}

So, what's going on here? I am not completely sure. QEMU looks quite reasonable to me. docker seems to have trouble spawning processes quickly which might mean that we should abandon the docker/windows ideas.

embray, what do you think?

comment:10 Changed 3 years ago by saraedum

  • Authors set to Julian Rüth
  • Milestone changed from sage-8.3 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

I propose to close this ticket as there is not really a point anymore in having these docker containers it seems.

comment:11 Changed 3 years ago by embray

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-wishlist

I think it's still desirable, and might be more feasible in the future. It's just not clear right now that it's feasible or useful.

comment:12 Changed 3 years ago by saraedum

I'm not even sure it's desirable. But sure, let's leave the ticket around.

comment:13 Changed 3 years ago by embray

I assume the Windows benchmarks imply also that the test is being run on Cygwin in Windows, right? I assume so but that wasn't exactly clear. Anyways, I'm not surprised it's so much slower in Docker. I'm not exactly sure how the native docker engine works on Windows, and it would be interesting to also benchmark various Windows system calls directly (some calls involved with Cygwin's fork include:

  • CreateProcessW (duh)
  • ReadProcessMemory
  • WriteProcessMemory
  • CreateEvent
  • WaitForSingleObject
  • CreatePipe

among numerous others).

I don't think we should waste a lot of time benchmarking Docker for Windows right now because there's nothing much we can do about it if it's slow.

My point is just that if there's even a little bit more overhead to system calls in a Windows native container, then that will add up even more for fork.

comment:14 Changed 3 years ago by saraedum

Yes, I was running cygwin inside the docker container.

Can you set this ticket back to "new" somehow?

comment:15 Changed 3 years ago by embray

  • Resolution set to fixed
  • Status changed from needs_review to closed

I think I need to close and reopen it

comment:16 Changed 3 years ago by embray

  • Resolution fixed deleted
  • Status changed from closed to new
Note: See TracTickets for help on using tickets.