Opened 9 years ago

Closed 8 years ago

#11851 closed task (fixed)

make pybtex an optional spkg

Reported by: ranosch Owned by: tbd
Priority: major Milestone: sage-5.2
Component: packages: optional Keywords: BibTeX citations
Cc: PolyBoRi, burcin Merged in: sage-5.2.beta0
Authors: Niels Ranosch, Burcin Erocal Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by ranosch)

See http://pybtex.sourceforge.net/ for a description.

Should be a standard spkg later, so that the citation system can use it.


Apply trac_11851-hgignore.patch to the scripts repo.

New optional spkg: http://ubuntuone.com/6iNB5C4qYc6UZWEwaAcOx1

Attachments (1)

trac_11851-hgignore.patch (389 bytes) - added by jhpalmieri 8 years ago.
SAGE_ROOT/local/bin repo

Download all attachments as: .zip

Change History (20)

comment:1 follow-up: Changed 9 years ago by jhpalmieri

When I install the spkg on a Mac OS X box, I see this

Processing PyYAML-3.10.zip
Running PyYAML-3.10/setup.py -q bdist_egg --dist-dir /var/folders/JV/JVYCpshdHd4FFoThuUgD8k+++TI/-Tmp-/easy_install-e_csNv/PyYAML-3.10/egg-dist-tmp-KAUGDh
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:2:18: error: yaml.h: No such file or directory
build/temp.macosx-10.6-i386-2.6/check_libyaml.c: In function ‘main’:
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: ‘yaml_parser_t’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: (Each undeclared identifier is reported only once
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: for each function it appears in.)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: expected ‘;’ before ‘parser’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:6: error: ‘yaml_emitter_t’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:6: error: expected ‘;’ before ‘emitter’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:8: warning: implicit declaration of function ‘yaml_parser_initialize’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:8: error: ‘parser’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:9: warning: implicit declaration of function ‘yaml_parser_delete’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:11: warning: implicit declaration of function ‘yaml_emitter_initialize’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:11: error: ‘emitter’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:12: warning: implicit declaration of function ‘yaml_emitter_delete’

libyaml is not found or a compiler error: forcing --without-libyaml
(if libyaml is installed correctly, you may need to
 specify the option --include-dirs or uncomment and
 modify the parameter include_dirs in setup.cfg)
zip_safe flag not set; analyzing archive contents...
Adding PyYAML 3.10 to easy-install.pth file

Are these errors important? The installation proceeds, apparently successfully, after this.

Second, would it be better to have three separate spkgs, one each for pybtex, PyYAML, and pyparsing?

Third, should the file patches/pybtex-doc_tree.patch.org be deleted? I see a reference to it in SPKG.txt, but why do we need it? It might be better just to give the link to the original patch and then say that it has been rebased. Rebasing is the only change, right?

comment:2 Changed 9 years ago by leif

  • Component changed from packages to optional packages
  • Description modified (diff)
  • Keywords BibTeX citations added
  • Status changed from new to needs_review

comment:3 in reply to: ↑ 1 Changed 9 years ago by ranosch

Thanks for looking into this!

Replying to jhpalmieri:

When I install the spkg on a Mac OS X box, I see this

Processing PyYAML-3.10.zip
Running PyYAML-3.10/setup.py -q bdist_egg --dist-dir /var/folders/JV/JVYCpshdHd4FFoThuUgD8k+++TI/-Tmp-/easy_install-e_csNv/PyYAML-3.10/egg-dist-tmp-KAUGDh
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:2:18: error: yaml.h: No such file or directory
build/temp.macosx-10.6-i386-2.6/check_libyaml.c: In function ‘main’:
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: ‘yaml_parser_t’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: (Each undeclared identifier is reported only once
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: for each function it appears in.)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:5: error: expected ‘;’ before ‘parser’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:6: error: ‘yaml_emitter_t’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:6: error: expected ‘;’ before ‘emitter’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:8: warning: implicit declaration of function ‘yaml_parser_initialize’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:8: error: ‘parser’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:9: warning: implicit declaration of function ‘yaml_parser_delete’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:11: warning: implicit declaration of function ‘yaml_emitter_initialize’
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:11: error: ‘emitter’ undeclared (first use in this function)
build/temp.macosx-10.6-i386-2.6/check_libyaml.c:12: warning: implicit declaration of function ‘yaml_emitter_delete’

libyaml is not found or a compiler error: forcing --without-libyaml
(if libyaml is installed correctly, you may need to
 specify the option --include-dirs or uncomment and
 modify the parameter include_dirs in setup.cfg)
zip_safe flag not set; analyzing archive contents...
Adding PyYAML 3.10 to easy-install.pth file

Are these errors important? The installation proceeds, apparently successfully, after this.

PyYAML Supports libYAML for speed reasons only, so you can ignore these messages. But I removed libYAML support from the SPKG (I will update the link soon), so that nobody gets confused.

Second, would it be better to have three separate spkgs, one each for pybtex, PyYAML, and pyparsing?

sagenb also includes its dependencies. It would be too much trouble to maintain three different SPKGs.

Third, should the file patches/pybtex-doc_tree.patch.org be deleted? I see a reference to it in SPKG.txt, but why do we need it? It might be better just to give the link to the original patch and then say that it has been rebased. Rebasing is the only change, right?

We don't know, if the link will change or will be unavailable some day. Burcin thought, it would probably be safer like this.

comment:4 Changed 9 years ago by ranosch

  • Description modified (diff)

Updated package not to support system-libYAML-bindings. Link is now http://sage.math.washington.edu/home/burcin/spkg/pybtex-0.15-r1.spkg

@jhpalmieri: Would you mind to look at it again, please?

The changes are available on https://bitbucket.org/niels_mfo/pybtex-0.15

comment:5 Changed 8 years ago by ranosch

  • Description modified (diff)

Upstream update. See SPKG.txt for more info.

comment:6 follow-up: Changed 8 years ago by jhpalmieri

This installed correctly, even though I disabled internet access while installing it. So that's good.

Regarding SPKG.txt: surely Python is a dependency. You should make this explicit, so that if/when this becomes a standard package, people where it goes in the makefile spkg/standard/deps. Also, I would add a little bit to the description; for example,

  • SPKG.txt

    diff --git a/SPKG.txt b/SPKG.txt
    a b  
    22
    33== Description ==
    44
    5 Pybtex is a drop-in replacement for BibTeX written in Python.
     5Pybtex is a drop-in replacement for BibTeX written in Python. That is,
     6you can run 'pybtex FILE' instead of 'bibtex FILE'.
    67
    78We use the bazaar trunk because it is less buggy and more feature-rich.
    89The latest version is available through this command:

When I ran this on a file, I got this error message:

amsplain.bst: %
amsplain.bst: ^^^
amsplain.bst: ERROR: Syntax error in line 333: BST command expected.

but maybe I don't have things configured correctly...

It might be a good idea to write an spkg-check script, which does nothing if nose is not installed, but runs self-tests if it is. This is not very important right now, but if you have a chance...

Finally, you need an attachment for the scripts repository. See attached.

Changed 8 years ago by jhpalmieri

SAGE_ROOT/local/bin repo

comment:7 Changed 8 years ago by jhpalmieri

  • Description modified (diff)

comment:8 in reply to: ↑ 6 Changed 8 years ago by ranosch

  • Description modified (diff)

Replying to jhpalmieri:

This installed correctly, even though I disabled internet access while installing it. So that's good.

Regarding SPKG.txt: surely Python is a dependency. You should make this explicit, so that if/when this becomes a standard package, people where it goes in the makefile spkg/standard/deps. Also, I would add a little bit to the description; for example, [...]

Thanks for checking, updated the spkg with your suggestions.

When I ran this on a file, I got this error message:

amsplain.bst: %
amsplain.bst: ^^^
amsplain.bst: ERROR: Syntax error in line 333: BST command expected.

but maybe I don't have things configured correctly...

What exactly did you do? I cannot reproduce the error.

It might be a good idea to write an spkg-check script, which does nothing if nose is not installed, but runs self-tests if it is. This is not very important right now, but if you have a chance...

I don't think that is possible. Maybe when nose becomes an optional package. But at the moment, when nose is installed on the system, sage can run the nosetests command, but python won't find nose:

Traceback (most recent call last):
  File "/usr/bin/nosetests", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/niels/mfo/sage-5.0/local/lib/python2.7/site-packages/distribute-0.6.16-py2.7.egg/pkg_resources.py", line 2672, in <module>
    working_set.require(__requires__)
  File "/home/niels/mfo/sage-5.0/local/lib/python2.7/site-packages/distribute-0.6.16-py2.7.egg/pkg_resources.py", line 655, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/niels/mfo/sage-5.0/local/lib/python2.7/site-packages/distribute-0.6.16-py2.7.egg/pkg_resources.py", line 553, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: nose==1.1.2

That error occurs when running nosetests in sage -sh or spkg-check with nose installed on my system.

Finally, you need an attachment for the scripts repository. See attached.

Thanks, didn't know about that.

comment:9 follow-up: Changed 8 years ago by jhpalmieri

Regarding nose, I was thinking of something like this in spkg-check:

if [ -x "$SAGE_LOCAL/bin/nosetests" ]; then
    # RUN TESTS HERE.
    if [ $? -ne 0 ]; then  # REPLACE WITH SUITABLE CHECK THAT TESTS PASSED.
        echo >&2 "Error running tests."
        exit 1
    fi
else
    echo "Not running tests since nose is not installed."
fi

Or you could replace the first line with a test for the suitable library in $SAGE_ROOT/local/lib/python/site-packages, say by running a little python script.

comment:10 in reply to: ↑ 9 Changed 8 years ago by ranosch

Am I missing something, or would this only make sense if nose would actually be a (at least optional) spkg. Sure, you could install nose into sage by hand. But that sounds a little hacky to me. I would much rather prefer to wait for the spkg. (Please correct me, if I'm missing something. Otherwise I feel like making (risking) more problems than solving.)

comment:11 follow-up: Changed 8 years ago by jhpalmieri

I still think that Python should be listed as a dependency. Otherwise, this looks okay.

comment:12 in reply to: ↑ 11 Changed 8 years ago by ranosch

Sure. Is that alright?

comment:13 follow-up: Changed 8 years ago by jhpalmieri

I'm happy with your changes, but, sorry, I keep finding more small problems. I think the last one is this: in the ChangeLog section of SPKG.txt, the changes should be in the reverse order, starting with the most recent, ending with the initial version.

comment:14 in reply to: ↑ 13 Changed 8 years ago by ranosch

Oh, that's right, the order needs to be reversed: https://bitbucket.org/niels_mfo/pybtex/src/20120618/SPKG.txt

I'm really happy for any communication whatsoever, as long as we all work on improving stuff. So if there any more problems (big or small), please don't hesitate to write about them :-)

comment:15 Changed 8 years ago by jhpalmieri

  • Reviewers set to John Palmieri

Yes, that looks good now. Regarding spkg-install, you don't really need to create and delete the directory tmp: I think you can just unzip and build !PyYAML-3.10.zip in place:

  • spkg-install

    diff --git a/spkg-install b/spkg-install
    a b if [ "x$SAGE_LOCAL" = x ]; then 
    66   exit 1
    77fi
    88
     9CUR=`pwd`
     10
    911# need to suppress confusing error messages and don't depend on system
    1012# libraries, so easy_install is not available
    11 mkdir tmp &&
    12     unzip -o PyYAML-3.10.zip -d tmp &&
    13     cd tmp/PyYAML-3.10 &&
     13unzip -o PyYAML-3.10.zip &&
     14    cd PyYAML-3.10 &&
    1415    python setup.py --without-libyaml install --prefix="$SAGE_LOCAL"
    1516if [ $? -ne 0 ]; then
    1617   echo "Error installing PyYAML."
    1718   exit 1
    1819fi
    1920
    20 cd ../..
    21 rm -rf tmp
     21cd "$CUR"
    2222
    2323easy_install pyparsing-1.5.6.zip
    2424if [ $? -ne 0 ]; then

But this is not very important; it works either way.

So if you upgrade the spkg to use your new version of SPKG.txt, you can switch this to "positive review".

comment:16 Changed 8 years ago by ranosch

  • Description modified (diff)
  • Status changed from needs_review to positive_review

comment:17 Changed 8 years ago by jdemeyer

  • Milestone changed from sage-5.1 to sage-5.2

comment:18 Changed 8 years ago by schilly

moved spkg to the servers

comment:19 Changed 8 years ago by jdemeyer

  • Merged in set to sage-5.2.beta0
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.