Opened 11 years ago

Closed 8 years ago

#6897 closed enhancement (invalid)

Migrate Notebook to a WSGI Framework (Django, Pylons, Werkzeug,...)

Reported by: timdumol Owned by: timdumol
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: notebook Keywords:
Cc: rkirov, mhansen Merged in:
Authors: Reviewers: Jason Grout
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by timdumol)

Moving Notebook to Django will abstract the server and user authentication, and give us a free admin interface, as well as a large support codebase.

Plan:

  • Replace guard.py and avatars.py with Django user authentication.
  • Replace twist.py with Django Views.
  • Change run_notebook.py to launch Twisted as a WSGI container.

1On Aug 31, 11:20 pm, William Stein <wst...@…> wrote:

<SNIP>

I'm not at all convinced that using twisted in any way (web or web2) is a good idea for the Sage notebook. I plan to revisit this in late September. The first thing I plan to do is consider switching from twisted to Django, as is done in codenode -- see http://codenode.org/-- hopefully, even sharing code with that project. Of course, twisted could still get used at a certain level behind the scenes, but the Sage notebook would then no longer explicitly use it.

So if you want to help as you describe above, perhaps you could accelerate this. This involves:

(1) getting familiar with Django, if you don't already know it.

(2) reading through the current codenode codebase

(3) then formulating a plan to replace server/notebook/twist.py with

something based either directly on Django, or possibly using codenode in some way.

The above is what I would do, but if you do it first that would be spectacular.

William

Change History (21)

comment:1 Changed 11 years ago by timdumol

  • Description modified (diff)

comment:2 Changed 11 years ago by timdumol

  • Owner changed from boothby to timdumol
  • Status changed from new to assigned

comment:3 Changed 11 years ago by mpatel

A naive question: Does Django support asynchronous responses?

comment:4 Changed 11 years ago by timdumol

If you mean AJAX (pull), then yes, Django supports AJAX. If you mean Comet (push) though, then no, as far as I can tell. From what I know, Comet can only be done with event-based servers (e.g., Twisted, Orbited). We can, however, let Twisted deal with async responses, while Django can deal with everything else -- as is done in Codenode. We still get the benefits of a pluggable database architecture, user authentication, url routing, etc.

comment:6 Changed 11 years ago by mpatel

Thanks very much for the explanation. I'm not familiar with Python web frameworks, so I don't have a strong opinion about what's best for Sage. I'm definitely not arguing against migrating to Django.

My interest stems from #6855, which I'd like to implement with Comet, if possible. I think I'll take a closer look at Twisted's deferreds. Is this possibly relevant?

By the way, what are your thoughts about FirePython? I think we can use the middleware part to send logging information to the browser. As far as I'm aware, Sage does not yet use Python's logging module, although #6187 uses it in the doc builder. Having a "unified" logging facility in the Sage library could be useful.

By another way, it appears there's a new, event-based server from Facebook called Tornado. I'm not sure yet why they didn't use Twisted.

comment:7 Changed 11 years ago by timdumol

I'm not too experienced in async programming, so I'm afraid I can't answer your question regarding deferreds. There is an abundance of tutorials on using Orbited for comet, but none that I see for Twisted. Code will be needed for the client in Javascript (I believe there are libraries for this), and for the server. Orbited can provide both -- although that will mean adding another package to Sage. It may be worth looking into it. Here's an example on using Twisted, Django and Orbited together.

FirePython looks great! It will be *very* handy. It seems to be easy to add once the Django work is done too.

The reasons for creating Tornado instead of Twisted are outlined here in the comments. The developers found Twisted's web support messy.

comment:8 Changed 11 years ago by mpatel

Thanks again. I'll try to investigate.

Can the migration subsume or simplify some of the tasks listed on the Sage Usability page? If so, please feel free to add comments. I think your input would be greatly appreciated.

comment:9 Changed 11 years ago by timdumol

Django migration should help with the ff. issues:

  • Streamlined deployment - account management, fewer bugs, etc.
  • LDAP Authentication.
  • Add users as admins. (fprimex)
  • User groups.

comment:10 Changed 11 years ago by timdumol

Apparently, we need the Twisted-trunk to run on Django. Twisted-trunk has some fixes that make it possible to use Django that have not been included in Twisted 8.2.0. Notably, POST requests don't seem to get passed on to Django in 8.2.0. I have uploaded spkg's at http://drop.io/trac_6897_reqs that are needed to run this. Jinja 2 is also reuired (which should not be a problem, since I expect #6586 to be merged in before this gets done).

Codenode also includes the Twisted-trunk in its package.

comment:11 Changed 11 years ago by timdumol

  • Summary changed from Migrate Notebook to Django to Migrate Notebook to a WSGI Framework (Django, Pylons, Werkzeug,...)

comment:12 Changed 11 years ago by was

Doesn't Twisted.web2 officially support WSGI? That was my naive understanding, though I haven't tested it. This is a good sign:

sage: import twisted.web2.wsgi
sage: help(twisted.web2.wsgi)
    class WSGIResource(__builtin__.object)
     |  A web2 Resource which wraps the given WSGI application callable.
     |  
     |  The WSGI application will be called in a separate thread (using
     |  the reactor threadpool) whenever a request for this resource or
     |  any lower part of the url hierarchy is received.
     |  
     |  Methods defined here:

comment:13 follow-up: Changed 11 years ago by timdumol

As far as I know, yes, Twisted.web2 does support WSGI. I have only used Twisted.web, though. I was/am under the impression that Twisted.web2 is not supported upstream anymore, and thus we should avoid using it.

comment:14 Changed 11 years ago by timdumol

  • Report Upstream set to N/A

Twisted 9.0 has finally been released, which means that WSGI is now supported by Twisted.web.

comment:15 in reply to: ↑ 13 Changed 11 years ago by was

Replying to timdumol:

As far as I know, yes, Twisted.web2 does support WSGI. I have only used Twisted.web, though. I was/am under the impression that Twisted.web2 is not supported upstream anymore, and thus we should avoid using it.

True. We should port the notebook back from Twisted.web2 to Twisted.web. This will be some significant work but it should get done.

comment:16 Changed 10 years ago by jason

Another point about performance of python options: http://nichol.as/asynchronous-servers-in-python

comment:17 Changed 10 years ago by jason

  • Cc rkirov mhansen added

This seems like a relevant ticket to the move to a flask-based notebook. Well, at least the title is relevant, even if most of the content is outdated now.

comment:18 Changed 9 years ago by mboratko

Was a decision made here regarding which python middleware to go with? Was flask decided on elsewhere (could you point me to where if that is the case)?

comment:19 Changed 9 years ago by jason

We are using flask in the "new" notebook server. This decision was made at least by Jan 2011; I don't know of a specific place to point you, but there have been lots of discussions on the sage-notebook mailing list about it. I think this ticket should be closed as it isn't helpful at all at this point. The ticket is too vague. I'm not sure who the release manager is right now, though, so I don't know who to CC.

comment:20 Changed 9 years ago by was

Jeroeon is the current release manager.

comment:21 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-feature to sage-duplicate/invalid/wontfix
  • Resolution set to invalid
  • Reviewers set to Jason Grout
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.