Opened 2 years ago

Closed 2 years ago

#18859 closed enhancement (fixed)

Change sage -i to install with dependencies -- documentation

Reported by: jdemeyer Owned by:
Priority: major Milestone: sage-6.9
Component: documentation Keywords:
Cc: Merged in:
Authors: Jeroen Demeyer Reviewers: Ralf Stephan, John Palmieri, Nathann Cohen
Report Upstream: N/A Work issues:
Branch: 4a10ed3 (Commits) Commit: 4a10ed3ed7e7cd03c67d89e94dde0bb5ff0039ba
Dependencies: #19101 Stopgaps:

Description (last modified by jdemeyer)

Change documentation after #19101.

Change History (54)

comment:1 Changed 2 years ago by jdemeyer

  • Branch set to u/jdemeyer/update_docs_to_use__make_pkgname__instead_of____sage__i_pkgname_

comment:2 Changed 2 years ago by jdemeyer

  • Commit set to f73faeb3e2cabd8017f83267db1d21543338cbdd
  • Status changed from new to needs_review

New commits:

f73faebUpdate docs to use "make pkgname" instead of "./sage -i pkgname"

comment:3 follow-up: Changed 2 years ago by ncohen

Yoooooooo Jeroen!

Shouldn't we expose this as a 'sage option' instead? This patch of yours replaces 'sage -i' everywhere, but you can type 'sage -i <something>' anywhere while you have to be in Sage's root folder to type 'make <something>'.

I expect that those who would have known how to read to the following message

raise TypeError("The optional arb package is not installed. "
                "Consider installing it via 'sage -i arb'")

may not all know how to react to

raise TypeError("The optional arb package is not installed. "
                "Consider installing it via 'make arb'")

for they would not immediately know where it should be typed.

Nathann

Last edited 2 years ago by ncohen (previous) (diff)

comment:4 in reply to: ↑ 3 Changed 2 years ago by jdemeyer

Replying to ncohen:

Yoooooooo Jeroen!

Shouldn't we expose this as a 'sage option' instead?

You mean something like sage --make arb?

comment:5 Changed 2 years ago by jdemeyer

Or a short-hand ./sage -m arb?

comment:6 follow-ups: Changed 2 years ago by tmonteil

Why not keeping ./sage -i arb ? If the current sage -i is superseded by make arb from SAGE_ROOT, then we could just make sage -i call $MAKE --directory=$SAGE_ROOT (unless sage -i is currently used by the Makefile) ?

comment:7 in reply to: ↑ 6 Changed 2 years ago by ncohen

Why not keeping ./sage -i arb ? If the current sage -i is superseded by make arb from SAGE_ROOT, then we could just make sage -i call $MAKE --directory=$SAGE_ROOT (unless sage -i is currently used by the Makefile) ?

+1.

Nathann

comment:8 in reply to: ↑ 6 ; follow-up: Changed 2 years ago by jdemeyer

Replying to tmonteil:

If the current sage -i is superseded by make arb from SAGE_ROOT

It's not superseded. I still want to support ./sage -i like before because

  1. ./sage -i works with old-style packages
  2. ./sage -i has options like -f, -s, -d
  3. Sometimes, it might be useful to "just" install a particular package, without all the build/install stuff.

comment:9 Changed 2 years ago by rws

  • Branch changed from u/jdemeyer/update_docs_to_use__make_pkgname__instead_of____sage__i_pkgname_ to public/18859

comment:10 Changed 2 years ago by rws

  • Commit changed from f73faeb3e2cabd8017f83267db1d21543338cbdd to 980fd2cdbf2644b957f9669132480dc60a946a93
  • Milestone changed from sage-6.8 to sage-6.9
  • Reviewers set to Ralf Stephan
  • Status changed from needs_review to positive_review

Positive because patchbot and me, we both are happy with this. The sage -m/--make is a feature of another ticket.


New commits:

980fd2cMerge branch 'u/jdemeyer/update_docs_to_use__make_pkgname__instead_of____sage__i_pkgname_' of trac.sagemath.org:sage into tmp5

comment:11 follow-up: Changed 2 years ago by ncohen

  • Status changed from positive_review to needs_work

Needs work because thi is making things more difficult for users. Besides, merely replacing 'sage -i' with 'make' will not do, as this obviously does not work from any directory.

comment:12 Changed 2 years ago by ncohen

P.S:

+<<<<<<< HEAD
     Working with sandpile divisors::
 
     sage: S = sandpiles.Complete(4)
@@ -310,6 +311,8 @@ Distribution of avalanche sizes::
     sage: D.weierstrass_rank_seq(0)
     (2, 1, 0, 0, 0, -1)
 
+To calculate linear systems associated with divisors, 4ti2 must be installed.
+The easiest way to do this is to run ``make 4ti2``.
 """

comment:13 in reply to: ↑ 11 ; follow-up: Changed 2 years ago by rws

  • Branch changed from public/18859 to u/jdemeyer/update_docs_to_use__make_pkgname__instead_of____sage__i_pkgname_
  • Commit changed from 980fd2cdbf2644b957f9669132480dc60a946a93 to f73faeb3e2cabd8017f83267db1d21543338cbdd

OOps wrong branch. As there is dissent I'll replace it with the previous.

Replying to ncohen:

Needs work because thi is making things more difficult for users.

This statement is in need of facts.

Besides, merely replacing 'sage -i' with 'make' will not do, as this obviously does not work from any directory.

I understood it's augmentation not replacement.

Whatever.

comment:14 in reply to: ↑ 13 ; follow-up: Changed 2 years ago by ncohen

Needs work because thi is making things more difficult for users.

This statement is in need of facts.

Seriously?

Okay...

Well: people here are trying to make it easy for evrybody to install this software. They create live USB key, binaries for all platforms, debian/gentoo/Mac versions of the software so that it can be installed in a couple of clicks. I'd be surprised if most of those users even know where SAGE_ROOT is given that they did not install it manually. Right now we have a "simple" way to install a package, i.e with "sage -i" (and I am sure that some of them do not even know how to open a console), and an easier one (that we should fix) through install_package(pkgname) in a Sage session.

Pray excuse me, but what exactly is the necessity of changing the user interface? We have total control on what 'sage -i' does, as it is one of our scripts. If you want it to have a different behaviour, if you want to add optional flags, if you want to change the default action, you *can* do it. So why this change, when all the messages you update now do not even contain enough information for the users to guess what they should do?

If the reason you want to keep simultaneously a 'sage -i' different from 'make' if to not check dependencies, I think that we can make it all work this way:

  • 'sage -i' -- we make it run the 'make' command
  • 'sage -i -f' -- we make it for the install of a package without checking its dependencies

You can add a '--no-dependencies' flag if you feel like it, but I really don't see why we should change the user interface for that. It's easier to adapt *our* usage then the user's.

Nathann

comment:15 in reply to: ↑ 8 ; follow-ups: Changed 2 years ago by jhpalmieri

Replying to jdemeyer:

Replying to tmonteil:

If the current sage -i is superseded by make arb from SAGE_ROOT

It's not superseded. I still want to support ./sage -i like before because

  1. ./sage -i works with old-style packages
  2. ./sage -i has options like -f, -s, -d
  3. Sometimes, it might be useful to "just" install a particular package, without all the build/install stuff.

Note that if #19004 gets merged, then item 1 is no longer completely true: to install an old-style package, I think you would have to download it manually and then run sage -i file.spkg.

Regarding point 3, I feel like the default behavior for sage -i should be to install the prerequisites, and there should be a flag (--no-deps?) to force an installation ignoring its dependencies. What is the downside to this?

comment:16 Changed 2 years ago by git

  • Commit changed from f73faeb3e2cabd8017f83267db1d21543338cbdd to 626113afdc2a3023f2ce1e83fa8d0123cbae442f

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

626113aUpdate docs to use "make pkgname" instead of "./sage -i pkgname"

comment:17 in reply to: ↑ 15 Changed 2 years ago by jdemeyer

Replying to jhpalmieri:

Note that if #19004 gets merged, then item 1 is no longer completely true: to install an old-style package, I think you would have to download it manually and then run sage -i file.spkg.

Then I don't like #19004...

My proposal is essentially to forget about ./sage -i and use make instead. Ordinary users should not run ./sage -i package, they should run make package. If they really know what they are doing, they can use ./sage -i package.

comment:18 follow-up: Changed 2 years ago by jdemeyer

  • Status changed from needs_work to needs_review

Setting back to needs_review since I don't quite understand the problem here...

comment:19 in reply to: ↑ 15 Changed 2 years ago by jdemeyer

Replying to jhpalmieri:

Regarding point 3, I feel like the default behavior for sage -i should be to install the prerequisites, and there should be a flag (--no-deps?) to force an installation ignoring its dependencies. What is the downside to this?

To answer your question with a question:

Regarding point 3, I feel like the default behavior for make should be to install the prerequisites, and there should be a more low-level command (./sage -i?) to force an installation ignoring its dependencies. What is the downside to this?

More seriously: why make ./sage -i more complicated than it needs to be? Instead of having ./sage -i run make, it makes a lot more sense to do it the other way around: make should run ./sage -i. The high-level interface should run the low-level command.

Last edited 2 years ago by jdemeyer (previous) (diff)

comment:20 in reply to: ↑ 18 Changed 2 years ago by ncohen

  • Status changed from needs_review to needs_work

Setting back to needs_review since I don't quite understand the problem here...

See 14. There is no need to change the user interface.

They currently have a simple way to install a package through sage -i, and unless you explain why you have to change this interface, I disagree with what this branch does.

comment:21 in reply to: ↑ 14 ; follow-up: Changed 2 years ago by jdemeyer

Replying to ncohen:

Pray excuse me, but what exactly is the necessity of changing the user interface?

To have one command (make) which takes care of everything regarding the Sage build. So this ticket is actually a simplification. Users won't need to remember another obscure command (./sage -i) to install packages.

comment:22 Changed 2 years ago by jdemeyer

  • Status changed from needs_work to needs_review

comment:23 in reply to: ↑ 21 ; follow-up: Changed 2 years ago by ncohen

To have one command (make) which takes care of everything regarding the Sage build. So this ticket is actually a simplification. Users won't need to remember another obscure command (./sage -i) to install packages.

What makes it impossible to have 'sage -i' run 'make' where it should be run?

comment:24 Changed 2 years ago by ncohen

  • Status changed from needs_review to needs_info

comment:25 in reply to: ↑ 23 ; follow-up: Changed 2 years ago by jdemeyer

Replying to ncohen:

What makes it impossible to have 'sage -i' run 'make' where it should be run?

It's not impossible, it's just a needless complication.

Analogy: you don't want gcc to keep track of dependencies, that's why you use make: usually, make runs gcc and not the other way around. Similarly, make should run ./sage -i and not the other way around.

comment:26 in reply to: ↑ 25 ; follow-up: Changed 2 years ago by ncohen

  • Status changed from needs_info to needs_work

It's not impossible, it's just a needless complication.

Analogy: you don't want gcc to keep track of dependencies, that's why you use make: usually, make runs gcc and not the other way around. Similarly, make should run ./sage -i and not the other way around.

My problem is not whether you should have a way to track dependencies and a way to not track them. My problem is that we have been using and advertising 'sage -i' for years. It appears in our doc, on some people's web page, and we can expect users to know how to run it (those who know how to open a console).

We don't have one year deprecations policies to change the behaviour of [some lost Sage function in a lost submodule] so that you can change 'sage -i' to 'make' without warning, especially when (see 12) we cannot expect all users to know what 'make' is, nor to know where 'SAGE_ROOT' is.

As it seems that there is no technical problem in changing the user interface, I set this ticket back to needs_work.

Nathann

Last edited 2 years ago by ncohen (previous) (diff)

comment:27 in reply to: ↑ 26 ; follow-up: Changed 2 years ago by jdemeyer

  • Work issues set to ???

Replying to ncohen:

My problem is not whether you should have a way to track dependencies and a way to not track them. My problem is that we have been using and advertising 'sage -i' for years. It appears in our doc, on some people's web page, and we can expect users to know how to run it

Yes, and ./sage -i will keep working exactly like before, which I consider a good thing (keeping in mind backwards compatibility). Why do you consider that a bad thing?

we cannot expect all users to know what 'make' is

Well, I don't think that running make foo is inherently more complicated than running sage -i foo. And if this is really the only argument against this ticket, we can easily add an option sage -m to run make.

comment:28 in reply to: ↑ 27 ; follow-up: Changed 2 years ago by ncohen

Yes, and ./sage -i will keep working exactly like before, which I consider a good thing (keeping in mind backwards compatibility). Why do you consider that a bad thing?

If it still works like before, why do you change all occurrences of 'sage -i' in the source and doc?

Well, I don't think that running make foo is inherently more complicated than running sage -i foo.

It is. If you just say "type 'make pkgname'" in a doc/exception message, I am 100% sure that people will open terminals, type what they are requested to type, and not know what to do with the message "There is no Makefile in this folder" just because they are not in SAGE_ROOT.

Nathann

comment:29 in reply to: ↑ 28 ; follow-up: Changed 2 years ago by jdemeyer

Replying to ncohen:

Yes, and ./sage -i will keep working exactly like before, which I consider a good thing (keeping in mind backwards compatibility). Why do you consider that a bad thing?

If it still works like before, why do you change all occurrences of 'sage -i' in the source and doc?

Because installing packages with dependencies is better than installing packages without dependencies. The old command still works, we are just documenting the high-level interface instead of the low-level command.

It is. If you just say "type 'make pkgname'" in a doc/exception message, I am 100% sure that people will open terminals, type what they are requested to type, and not know what to do with the message "There is no Makefile in this folder" just because they are not in SAGE_ROOT.

OK fine, I see your point. Would you agree with this ticket if we added an option sage -m to run make and documentated that?

comment:30 follow-up: Changed 2 years ago by jdemeyer

Do you understand the analogy between

  1. When building some package containing C code, usually you run make which runs individual gcc commands, keeping in mind dependencies between files.
  2. When building Sage, usually you run make which runs individual sage -i commands, keeping in mind dependencies between packages.

Having sage -i run make makes little sense, just like having gcc run make makes little sense.

comment:31 in reply to: ↑ 29 ; follow-up: Changed 2 years ago by ncohen

Because installing packages with dependencies is better than installing packages without dependencies. The old command still works, we are just documenting the high-level interface instead of the low-level command.

So why don't you just change the behaviour of the current command 'sage -i' so that it installs packages with dependencies? This would mean that the user interface does not need to be changed.

OK fine, I see your point. Would you agree with this ticket if we added an option sage -m to run make and documentated that?

It would mean that the users can actually hope to run the command that we give them, which is better. But there still isn't any reason that I see to tell them to use sage -m <pkgname> instead of sage -i <pkgname>. Let us adapt sage -i, which is currently used, to run what should be run.

comment:32 in reply to: ↑ 30 ; follow-up: Changed 2 years ago by ncohen

  1. When building Sage, usually you run make which runs individual sage -i commands, keeping in mind dependencies between packages.

Having sage -i run make makes little sense, just like having gcc run make makes little sense.

For this reason, a way out would be to have: 1) sage -i --nodep # install without taking dependencies into account. That would be what 'make' calls when building Sage on each individual package 2) sage -i # that would call make, and thus take dependencies into account.

Thus, running 'sage -i' would call make, which in turn would call 'sage -i --nodep' individually on the packages that need be installed. And this way, the user interface does not change. And this way, dependencies will be taken into account by default.

Nathann

comment:33 in reply to: ↑ 31 ; follow-up: Changed 2 years ago by jdemeyer

Replying to ncohen:

So why don't you just change the behaviour of the current command 'sage -i' so that it installs packages with dependencies?

I already answered this: for backwards compatibility and to not add unneeded complexity to that command.

This would mean that the user interface does not need to be changed.

I don't get this...

This ticket changes absolutely no interface, only documentation.

comment:34 Changed 2 years ago by jdemeyer

To understand each other better, could you please answer the following questions with yes/no:

  1. Should ./sage -b also build the dependencies of the Sage library?
  2. Should ./sage -docbuild ... also build the dependencies of the documentation?
  3. Should ./sage -i package ... also build the dependencies of package?

If the answer is not 3 times the same, I would like to know the reason.

comment:35 in reply to: ↑ 32 Changed 2 years ago by jdemeyer

Replying to ncohen:

And this way, the user interface does not change.

See comment 33: this ticket does not change any interface, only documentation.

And this way, dependencies will be taken into account by default.

The whole point of this ticket is that make package should be considered the default way of installing packages. So dependencies are taken into account by default.

Last edited 2 years ago by jdemeyer (previous) (diff)

comment:36 follow-up: Changed 2 years ago by jhpalmieri

I don't quite find backwards compatibility a compelling enough reason to keep the old behavior of sage -i ...: I find it likely that more users are currently misinstalling packages by not installing dependencies vs. ones who might want to force-install without a dependency-check in the future. The latter are more likely to know what they're doing and thus will be able to adapt to a change.

Given the amount of debate on this ticket, maybe we should have a discussion on sage-devel: how should we document installing packages?

  • make pkg
  • sage -i pkg (backwards-incompatible, with a new option sage -i --no-deps pkg)
  • sage -m pkg and/or sage --make pkg

(Or was there a discussion that I missed?) I have a slight, not strong, preference, for the second one. An advantage of 1 and 3 is that they keep the current behavior of sage -i. An advantage of 2 and 3 is that they can be run from anywhere, not just SAGE_ROOT.

As far as this ticket actually goes, if we want to recommend make pkg, then you should also change the sage command so that -i is not listed as a basic option. Same with src/doc/en/reference/repl/options.rst.

comment:37 in reply to: ↑ 33 ; follow-up: Changed 2 years ago by ncohen

So why don't you just change the behaviour of the current command 'sage -i' so that it installs packages with dependencies?

I already answered this: for backwards compatibility and to not add unneeded complexity to that command.

I do not think that backward compatibility matters here. If the behaviour of sage -i becomes something that the users should not call anymore, then it is better to change its behaviour so that they can keep calling it.

This would mean that the user interface does not need to be changed.

I don't get this...

This ticket changes absolutely no interface, only documentation.

By 'user interface' I mean 'what the users should call'. If you change the errors messages and the doc to tell the users to call something else, then surely our "interface with the users" changes.

To understand each other better, could you please answer the following questions with yes/no: Should ./sage -b also build the dependencies of the Sage library?

Not the subject here.

Should ./sage -docbuild ... also build the dependencies of the documentation?

Not the subject here.

Should ./sage -i package ... also build the dependencies of package?

Yes

If the answer is not 3 times the same, I would like to know the reason.

Is it really a problem for you if one command takes dependencies into account and not the others? If it is, it is not for me and the users probably do not care: among those three commands, they only run the third.

The whole point of this ticket is that make package should be considered the default way of installing packages. So dependencies are taken into account by default.

And for this reason it would be better, in order to *not* change our users's custom, to change what they have to call (sage -i) but rather to change the behaviour of that command (sage -i) so that it does what you want, i.e. take dependencies into account.

comment:38 in reply to: ↑ 36 ; follow-up: Changed 2 years ago by jdemeyer

Replying to jhpalmieri:

I don't quite find backwards compatibility a compelling enough reason to keep the old behavior of sage -i ...

It's not just that. It's also avoiding needless complexity. I really do not see why we should add another option to the sage -i script to do something which is really the job of make.

And what is your opinion on 34?

comment:39 in reply to: ↑ 37 Changed 2 years ago by jdemeyer

I'm writing to sage-devel...

comment:40 in reply to: ↑ 38 Changed 2 years ago by jhpalmieri

Replying to jdemeyer:

And what is your opinion on 34?

All things being equal, the answer to all three would be "no". Also, all three options for the sage command should be considered "advanced" options and not widely advertised.

Since sage -i has been the only way to install packages until recently, though, I'm not sure all things are equal. Maybe you're right, though, and we should encourage using make pkg, and after a while, sage -i pkg will be used less and less, and it can be thought of the same way as sage -b and sage --docbuild.

comment:41 follow-up: Changed 2 years ago by jdemeyer

Thierry Monteil has suggested to:

  1. change ./sage -i to install with dependencies
  2. add a new option, say ./sage -p which would work like the old ./sage -i (install a single package)

I think that suggestion might be an excellent compromise. Nathann, what do you think?

comment:42 in reply to: ↑ 41 ; follow-up: Changed 2 years ago by ncohen

I think that suggestion might be an excellent compromise. Nathann, what do you think?

I agree.

Honestly, I have no idea why this seems acceptable to you now and not yesterday. I don't see the difference between this and what we have been asking all along.

This is a wonder to me but well... Yes, yes, of course I agree. 'sage -i' installs a package with dependencies, and you prefer '-p' to '-i --no-deps'. No problem.

Nathann

comment:43 in reply to: ↑ 42 Changed 2 years ago by jdemeyer

Replying to ncohen:

Honestly, I have no idea why this seems acceptable to you now and not yesterday. I don't see the difference between this and what we have been asking all along.

Good then.

For me, there is a large difference between sage -i --no-deps and sage -p.

I won't have time immediately to work on it, so consider this ticket "on hold" for now...

comment:44 Changed 2 years ago by jdemeyer

  • Description modified (diff)
  • Summary changed from Update docs to use "make pkgname" instead of "./sage -i pkgname" to Change sage -i to install with dependencies -- documentation

I'll keep this ticket just for the documentation. The changes to the sage -i script are for #19101.

comment:45 Changed 2 years ago by jdemeyer

  • Description modified (diff)
  • Work issues ??? deleted

comment:46 Changed 2 years ago by git

  • Commit changed from 626113afdc2a3023f2ce1e83fa8d0123cbae442f to 4d1a42a073280bb29c6224e1e046d97f6b3ce1c1

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

4d1a42aUpdate docs with new commands to install packages

comment:47 Changed 2 years ago by jdemeyer

  • Dependencies set to #19101
  • Status changed from needs_work to needs_review

comment:48 Changed 2 years ago by git

  • Commit changed from 4d1a42a073280bb29c6224e1e046d97f6b3ce1c1 to be4762d5b155519e0fa1ef0752e4aa50f00636dc

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

be4762dUpdate docs with new commands to install packages

comment:49 Changed 2 years ago by jdemeyer

  • Description modified (diff)

comment:50 Changed 2 years ago by git

  • Commit changed from be4762d5b155519e0fa1ef0752e4aa50f00636dc to 4a10ed3ed7e7cd03c67d89e94dde0bb5ff0039ba

Branch pushed to git repo; I updated commit sha1. New commits:

4a10ed3Fix wrong uses of "sage -f" also

comment:51 Changed 2 years ago by jhpalmieri

This looks good to me now. Anyone else have any issues, or can we set it to "positive review"?

comment:52 Changed 2 years ago by ncohen

None from me, it looks good indeed.

comment:53 Changed 2 years ago by jdemeyer

  • Reviewers changed from Ralf Stephan to Ralf Stephan, John Palmieri, Nathann Cohen
  • Status changed from needs_review to positive_review

Thanks!

comment:54 Changed 2 years ago by vbraun

  • Branch changed from u/jdemeyer/update_docs_to_use__make_pkgname__instead_of____sage__i_pkgname_ to 4a10ed3ed7e7cd03c67d89e94dde0bb5ff0039ba
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.