Opened 8 months ago

Last modified 3 weeks ago

#33725 new enhancement

Migrate wiki.sagemath.org and trac.sagemath.org wiki contents to more suitable places

Reported by: Matthias Köppe Owned by:
Priority: critical Milestone: sage-9.8
Component: website/wiki Keywords:
Cc: Samuel Lelièvre, Dima Pasechnik, Thierry Monteil, Frédéric Chapoton, Julian Rüth, Kwankyu Lee, William Stein, Harald Schilly, John Palmieri Merged in:
Authors: Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

The problem with wiki.sagemath.org:

It excludes new and newish developers who do not have the required legacy Trac account but log in to Trac using their GitHub accounts.

Wikis work because of the principle of crowdsourcing. Without a crowd, the material will just become outdated. New material added is indistinguishable from old material.

There is also a conflict between viewing the Wiki (1) as an archive (as a record of past achievements, and perhaps for the benefit of future software archaeologists); (2) as something that is useful for current users and developers; and (3) a place for discussion and planning actions.

These uses are not compatible. We propose to separate these types of material clearly and move each to a more appropriate system.

Source material on wiki.sagemath.org:


Proposed targets:

Migration to Trac tickets / GitHub issues

Ticket/issue descriptions can be used for collecting/curating/staging material that eventually should be added to our documentation or to our code.

Migration to our documentation

(needs conversion to rst)

Current drawback: The documentation is only updated one every stable release (to be fixed in #33862)

Migration to www.sagemath.org

(needs conversion to html; or website scripts need extension to handle markdown or moinmoin sources; https://python-markdown.github.io/reference/; https://github.com/github/markup)


Considered but won't be needed:

Migration to GitHub wiki:

"Note: Search engines will not index the contents of wikis. To have your content indexed by search engines, you can use GitHub Pages in a public repository." https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis

Migration to GitHub pages:

https://pages.github.com/

(needs conversion to markdown)

Unclear why we should create a new website when we already have www.sagemath.org

Migration to trac.sagemath.org/wiki:

As proposed in https://groups.google.com/g/sage-devel/c/ayOL8_bzOfk/m/wOuVdXwbAgAJ, https://groups.google.com/g/sage-devel/c/Acy5nNZWaUA/m/DuqGh3JkCwAJ and https://groups.google.com/g/sage-devel/c/4ysVRpYGU_E

The Trac wiki markup appears to understand most of the old Sage wiki's markup, except:

  • In section headers, CamelCase words need to be escaped with "!"
  • edit the markup for included images
    • {{https://patchbot.sagemath.org/svg/Pending||width=48}} changes to [[Image(https://patchbot.sagemath.org/svg/Pending, width=48)]]
  • Replace <<TableOfContents>> by [[PageOutline]]

Also we can simplify some markup by replacing links to Trac tickets by just #....

  • A command that helps with this:
    sed 's/<<TableOfContents[()]*>>/[[PageOutline]]/;s,\[\ 
    [https://trac.sagemath.org/ticket/\([0-9]*\)|#\1\]\],#\1,g;/^=/s/ \ 
    ([A-Z][a-z][a-z]*[A-Z]\)/ !\1/g;' INPUT > OUTPUT
    
  • https://trac-hacks.org/wiki/MoinToTracScript (untested)

See, however, #30363 (Proposed migration from Trac to GitHub).


Format conversion tools, workflow:

  • To do while migrating: We also audit/update our documentation regarding links to the wiki.
    $ git grep 'wiki[.]sage'
    

References:

Change History (70)

comment:1 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:2 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:3 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:4 Changed 8 months ago by Thierry Monteil

Please don't. In terms of infrastructure, the wiki is pretty well hosted, while the VM that hosts the trac+wiki is unmaintained, so this looks like a regression to me (at least right now).

comment:5 Changed 8 months ago by Dima Pasechnik

I think that trac is better, as it allows contributions with github logins. And the bus factor of the present wiki hosting is rather small.

comment:6 in reply to:  4 Changed 8 months ago by Matthias Köppe

It's critical that we get rid of the old Sage wiki. As Dima says - it excludes new developers.

Trac is maintained because it has to be (thanks to all who keep it running!) The wiki part of it places no additional burden on maintaining Trac.

comment:7 Changed 8 months ago by Matthias Köppe

Priority: majorcritical

comment:8 Changed 8 months ago by Matthias Köppe

Cc: Frédéric Chapoton added
Description: modified (diff)

comment:9 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:10 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:11 Changed 8 months ago by Matthias Köppe

Cc: Julian Rüth added

comment:12 Changed 8 months ago by Matthias Köppe

Description: modified (diff)

comment:13 Changed 8 months ago by Matthias Köppe

Branch: u/mkoeppe/migrate_wiki_sagemath_org_to_trac_sagemath_org_wiki

comment:14 Changed 8 months ago by git

Commit: 57fc849fc561584a8ae726ed6d30162c003e1445

Branch pushed to git repo; I updated commit sha1. New commits:

57fc849configure.ac: Remove outdated advice regarding exotic platforms

comment:15 Changed 8 months ago by Matthias Köppe

Dependencies: #33088

comment:16 Changed 8 months ago by git

Commit: 57fc849fc561584a8ae726ed6d30162c003e144545c1a394ae75fad31da2d85e9a3e1481f37540a6

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

b46c953README: Document --enable-editable and --enable-download-from-upstream-url
a3ffe84README.md: Instructions for Apple Silicon
d7708dfREADME.md: Restructure macOS prep instructions
2e1d22dsrc/doc/en/installation/source.rst: Remove instructions for Solaris, AIX, HP-UX
9ca39a2README.md: Point to developer guide for git details
f1e1166src/doc/en/installation/source.rst: Point to README.md for step-by-step instructions
3f6c6e0src/doc/en/developer/workspace.rst: Link to Trac #30500
8716d80src/doc/en/installation/source.rst: Update required venv Python extension modules from build/pkgs/python3/spkg-configure.m4
10c9b87Merge #33088
45c1a39src/doc/en/developer/trac.rst: Update wiki URL for patchbot

comment:17 Changed 7 months ago by Matthias Köppe

Some pages, for example https://wiki.sagemath.org/SageMathExternalPackages, are written in RST.

The Trac wiki also supports RST - https://trac.sagemath.org/wiki/WikiRestructuredText - but it looks like it is not activated.

@dimpase, could you install docutils (as per https://trac.sagemath.org/wiki/WikiRestructuredText#Requirements)?

comment:18 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:19 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:20 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:21 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:22 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:23 Changed 7 months ago by Matthias Köppe

Description: modified (diff)

comment:24 Changed 7 months ago by git

Commit: 45c1a394ae75fad31da2d85e9a3e1481f37540a6da69787e997cefccb398a6dee9258ecfda3a26d0

Branch pushed to git repo; I updated commit sha1. New commits:

da69787src/doc: Fix 'hardcoded link could be replaced'

comment:25 Changed 7 months ago by Matthias Köppe

Cc: Kwankyu Lee added
Description: modified (diff)

comment:26 Changed 7 months ago by Matthias Köppe

@tmonteil, would you be able to provide me with a snapshot of the moinmoin wiki data, so I can try out the script https://trac-hacks.org/wiki/MoinToTracScript?

comment:27 Changed 7 months ago by Matthias Köppe

Milestone: sage-9.6sage-9.7

comment:28 Changed 3 months ago by Matthias Köppe

Milestone: sage-9.7sage-9.8

comment:29 in reply to:  4 Changed 3 months ago by Thierry Monteil

Replying to Thierry Monteil:

Please don't. In terms of infrastructure, the wiki is pretty well hosted, while the VM that hosts the trac+wiki is unmaintained, so this looks like a regression to me (at least right now).

comment:30 in reply to:  26 Changed 3 months ago by Matthias Köppe

Replying to Matthias Köppe:

@tmonteil, would you be able to provide me with a snapshot of the moinmoin wiki data, so I can try out the script https://trac-hacks.org/wiki/MoinToTracScript?

Do you have response to this other than repeating that you're against it?

comment:31 Changed 3 months ago by Matthias Köppe

Cc: William Stein added

comment:32 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:33 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:34 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:35 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:36 Changed 3 months ago by Matthias Köppe

Description: modified (diff)
Summary: Migrate wiki.sagemath.org to trac.sagemath.org/wikiMigrate wiki.sagemath.org to a more suitable place

comment:37 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:38 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:39 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:40 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:41 Changed 3 months ago by Matthias Köppe

Summary: Migrate wiki.sagemath.org to a more suitable placeMigrate wiki.sagemath.org contents to more suitable places

comment:42 Changed 3 months ago by Matthias Köppe

Cc: Harald Schilly added
Description: modified (diff)

comment:43 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:44 Changed 3 months ago by Matthias Köppe

Branch: u/mkoeppe/migrate_wiki_sagemath_org_to_trac_sagemath_org_wiki
Commit: da69787e997cefccb398a6dee9258ecfda3a26d0
Dependencies: #33088

comment:45 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:46 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:47 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:48 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:49 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:50 Changed 3 months ago by Tobias Diez

What's the reason why a wiki is necessary in addition to the main website and the documentation in the main repo? I think this leads only to possible duplication and confusion on where to put what.

By the way, sphinx accepts also markdown as input: https://www.sphinx-doc.org/en/master/usage/markdown.html

comment:51 Changed 3 months ago by William Stein

What's the reason why a wiki is necessary in addition to the main website and the documentation in the main repo?

In the past the wiki was very useful for organizing Sage Days and generally doing things that required quick updates by many people. I think there are much better tools for such activities these days.

comment:52 in reply to:  50 Changed 3 months ago by Matthias Köppe

Replying to Tobias Diez:

What's the reason why a wiki is necessary in addition to the main website and the documentation in the main repo? I think this leads only to possible duplication and confusion on where to put what.

Yes, I agree with you. My proposal is that we can move everything to (1) Trac tickets / GitHub issues, (2) our documentation, (3) www.sagemath.org, and nothing else is needed.

comment:53 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:54 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:55 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:56 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:57 Changed 3 months ago by Matthias Köppe

Description: modified (diff)

comment:58 Changed 2 months ago by Kwankyu Lee

Assuming github migration, Github Pages may be used to build a website to host wiki contents. It seems this website may be maintained by our developer community just like our sage repo.

Just an idea, as I have zero experience with all these. What do you think?

comment:59 in reply to:  58 Changed 2 months ago by Kwankyu Lee

Replying to Kwankyu Lee:

Assuming github migration, Github Pages may be used to build a website to host wiki contents. It seems this website may be maintained by our developer community just like our sage repo.

Ah... This is not different using the current sagemath website as this is in github as well.

Sorry for noise.

comment:60 Changed 2 months ago by Matthias Köppe

I went through the same steps of thinking. The only thing that's missing is support for .md files in our website scripts.

comment:61 in reply to:  60 Changed 2 months ago by Kwankyu Lee

Replying to Matthias Köppe:

The only thing that's missing is support for .md files in our website scripts.

That is an important point. Together with liberal write permission to developers, that will make a search-engine-indexed website close to a wiki site...

comment:62 Changed 2 months ago by Dima Pasechnik

one can hopefully have the content versioned in git (on GitHub), and then updating automatically, via GitHub Actions, is doable (it's not instant WYSIWYG, but OK)

comment:63 Changed 2 months ago by Matthias Köppe

Yes, the source code of our website is maintained on github. https://github.com/sagemath/website

I agree, it would be good to add GH Actions that runs what Harald currently runs manually after merging PRs, with a preview of the generated HTML on PRs.

comment:64 Changed 2 months ago by Matthias Köppe

Description: modified (diff)

comment:65 Changed 2 months ago by Matthias Köppe

Description: modified (diff)

comment:66 Changed 2 months ago by Kwankyu Lee

Cc: John Palmieri added

If we end up running self-hosted gitlab (for mirroring, archiving, or replacing github) instead of trac, then we can move wiki to the gitlab as it also has wiki system.

comment:67 Changed 7 weeks ago by Matthias Köppe

Description: modified (diff)

comment:68 Changed 3 weeks ago by Matthias Köppe

Cc: John Palmieri added; John Palmieri removed
Description: modified (diff)

comment:69 Changed 3 weeks ago by Matthias Köppe

Description: modified (diff)
Summary: Migrate wiki.sagemath.org contents to more suitable placesMigrate wiki.sagemath.org and trac.sagemath.org wiki contents to more suitable places

comment:70 Changed 3 weeks ago by Matthias Köppe

Description: modified (diff)
Note: See TracTickets for help on using tickets.