Ticket #3905: prog.patch
File prog.patch, 160.2 KB (added by , 11 years ago) |
---|
-
commontex/macros-new.tex
# HG changeset patch # User J. H. Palmieri <palmieri@math.washington.edu> # Date 1219199147 25200 # Node ID 8f9275ea4358f44711241bbc022128ef6a46d202 # Parent bba79837cb391923027900edb059116505472ab7 revising programming guide diff -r bba79837cb39 -r 8f9275ea4358 commontex/macros-new.tex
a b 45 45 % when run through latex2html. See the command \newbf above for an 46 46 % application of this. See the latex2html docs for more information. 47 47 \usepackage{html} 48 49 \usepackage{verbatim} 50 % the following is used for nesting verbatim environments, used in prog/prog.tex 51 \newenvironment{newverbatim}% 52 \verbatim% 53 {\endverbatim} 48 54 49 55 \newcommand{\by}[1]{\par\noindent{\bf AUTHOR:} #1\par} 50 56 -
prog/prog.tex
diff -r bba79837cb39 -r 8f9275ea4358 prog/prog.tex
a b 1 % see 3393, 3882 2 3 % coercion section 4 % mutability section should be rewritten, expanded. 5 % benchmarking? 6 1 7 \documentclass{manual} 2 8 \usepackage{textcomp} 3 \include{macros} 9 10 \include{macros-new} 4 11 % a hack to get around another hack. 5 12 \ifpdf 6 13 \usepackage{url} 7 14 \else 8 15 \fi 9 16 10 11 17 %\newcommand{\todo}[1]{\footnote{1}} 12 18 \newcommand{\todo}[1]{} 13 19 14 20 \newcommand{\swlist}{GAP, GSL, Matplotlib, Maxima, MWRANK, NetworkX, 15 NTL, Numpy, PARI and Singular }21 NTL, Numpy, PARI and Singular\xspace} 16 22 17 23 \usepackage{color} 18 24 \definecolor{hue}{rgb}{.202, .602, .58} … … 31 37 \begin{document} 32 38 33 39 34 \title{\SAGE ProgrammingGuide}40 \title{\SAGE Developer's Guide} 35 41 36 \author{William Stein%\thanks{Associate Professor, University of Washington, wstein@math.washington.edu.} 37 \ \ and David Joyner%\thanks{Mathematics Department, US Naval Academy, wdj@usna.edu} 38 } 42 \author{William Stein and David Joyner} 39 43 40 44 \maketitle 41 45 … … 48 52 \begin{abstract} 49 53 50 54 \noindent 51 Absolutely {\bf everybody} who uses \SAGE should contribute something 55 \SAGE is a free mathematics software system. It is implemented using 56 Python, Cython, and C++, and uses \swlist. It is free and open 57 source, and is available under the terms of the GNU Public License. 58 59 Everybody who uses \SAGE should contribute something 52 60 back to \SAGE at some point. Implement a new function, add examples 53 61 to the documentation, find bugs and typos, fix a bug, create a new 54 62 class, create a fast new C library, etc. This book is a guide about 55 how to contribute code backto \SAGE.63 how to contribute to \SAGE. 56 64 57 This document describes how to write programs using \SAGE, and also 58 how to modify and extend the core \SAGE libraries. 59 60 \SAGE stands for Software for Algebra and Geometry Experimentation. 61 It is implemented using Python, Cython, and C++, and uses \swlist. 62 It is free and open source, and is available under the terms of the 63 GNU Public License. 65 This document describes how to write programs using \SAGE, 66 how to modify and extend the core \SAGE libraries, and how to modify 67 \sage's documentation. It also discusses how to share your new and 68 modified code with other \SAGE users. 64 69 65 70 \end{abstract} 66 71 67 72 68 73 \tableofcontents 69 74 70 \todo{Remove I and me} 75 76 \part{Writing Code for \sage} 71 77 72 78 \chapter{Introduction} 73 79 If there is something you would like to implement and make … … 93 99 contains sophisticated, optimized number theory algorithms. Notably 94 100 absent from this triad is a good system for exact linear algebra 95 101 (something MAGMA does extremely well), but this gap is being filled by 96 code being written for \SAGE. \todo{Say something about GSL, 97 Matplotlib, Maxima, MWRANK, NetworkX, NTL, Numpy} 98 %Unfortunately, Singular, GAP, and PARI 99 %have vastly different languages and conventions. 102 code written for \SAGE. 103 \todo{Say something about GSL, Matplotlib, Maxima, MWRANK, NetworkX, 104 NTL, Numpy} 100 105 101 \SAGE is not just about gathering together functionality (something102 UNIX already does well), it is aboutproviding a clear, systematic and106 \SAGE is not just about gathering together functionality, it is about 107 providing a clear, systematic and 103 108 consistent way to access a large number of algorithms, in a coherent 104 109 framework that makes sense mathematically. In the design of \SAGE, the 105 110 semantics of objects, the definitions, etc., are informed by how the … … 108 113 This document was authored by William Stein, David Joyner and others 109 114 with the editorial help of Iftikhar Burhanuddin. 110 115 111 \chapter{\SAGE{} Coding Conventions}\label{ch:conventions}112 To meet the goal of making \sage easy to read, all Python/Cython code113 that is included with \sage should adhere to the style conventions114 mentioned in this chapter.115 116 116 \section{Coding Conventions} 117 Follow the standard Python formatting rules when writing code for Sage, 118 as explained at \url{http://www.python.org/doc/essays/styleguide.html}. 117 118 \chapter{Conventions for Coding in \SAGE}\label{ch:conventions} 119 120 To meet the goal of making \sage easy to read, maintain, and improve, 121 all Python/Cython code that is included with \sage should adhere to 122 the style conventions discussed in this chapter. 123 124 \section{Python Coding Conventions} 125 126 Follow the standard Python formatting rules when writing code for \sage, 127 as explained at 128 \url{http://www.python.org/dev/peps/pep-0008/} and 129 \url{http://www.python.org/dev/peps/pep-0257/}. 119 130 In particular, 120 131 \begin{itemize} 121 132 \item Use 4 spaces for indentation levels. Do not use tabs as they … … 128 139 \begin{verbatim} 129 140 def set_some_value() 130 141 \end{verbatim} 131 instead of :142 instead of CamelCase: 132 143 \begin{verbatim} 133 144 def SetSomeValue() 134 145 \end{verbatim} … … 137 148 \end{itemize} 138 149 139 150 140 \section{File and Directory names} 151 \section{File and Directory Names} 152 141 153 Python \sage library code uses the following conventions. Directory 142 154 names may be plural (e.g., \code{rings}) and file names are almost 143 155 always singular (e.g., \code{polynomial\_ring.py}). Note that the 144 156 file \code{polynomial\_ring.py} might still contain definitions of 145 157 several different types of polynomial rings. 146 158 147 \note{You are stronglyencouraged159 \note{You are encouraged 148 160 to include miscellaneous notes, emails, design 149 161 discussions, etc., in your package. Make these 150 162 plain text files (with extension \code{.txt}) 151 in a subdirectory directory called \code{notes}. 163 in a subdirectory called \code{notes}. (For example, see 164 \code{SAGE_ROOT/devel/sage/sage/ext/notes/}.) 152 165 } 153 166 154 %\section{Ordering of function names in a class}155 %Function names156 %should be defined in alphabetical order, with \code{__}'s first,157 %then \code{_}'s, then public functions. This is for easy of readability158 %and finding code. It also encourages functions with related behavior159 %to be named in a similar way, which makes it easier to find them160 %in the documentation and when using tab completion from the interpreter.161 162 167 163 168 \section{Headings of \SAGE Library Code Files} 164 The top of each \sage code file should appear as follows\todo{What is \section{Tutorial} below}: 169 170 The top of each \sage code file should follows this format: 165 171 166 172 \begin{verbatim} 167 173 r""" … … 171 177 ... 172 178 173 179 AUTHORS: 174 - YOUR NAME (2005-01-03): initial version175 - person (date in ISO year-month-day format): short desc180 -- YOUR NAME (2005-01-03): initial version 181 -- person (date in ISO year-month-day format): short desc 176 182 ... 177 - person (date in ISO year-month-day format): short desc183 -- person (date in ISO year-month-day format): short desc 178 184 179 \section{Tutorial}180 185 ... 181 186 182 187 Lots and lots of examples. 183 188 """ 184 189 185 190 #***************************************************************************** 186 # Copyright (C) 2006 William Stein <wstein@gmail.com> 187 # 2006 YOUR NAME <your email> 191 # Copyright (C) 2008 YOUR NAME <your email> 188 192 # 189 193 # Distributed under the terms of the GNU General Public License (GPL) 190 194 # http://www.gnu.org/licenses/ 191 195 #***************************************************************************** 192 196 \end{verbatim} 193 197 194 The following is t op of the ssmod.py file, which contains the195 SupersingularModule code and is located at 196 \code{SAGE_ROOT/devel/sage/sage/modular/ssmmod/ssmod.py}.198 The following is the top of the file 199 \code{SAGE_ROOT/devel/sage/sage/modular/ssmmod/ssmod.py}, which 200 contains the Supersingular Module code. 197 201 198 202 {\small 199 203 200 204 \begin{verbatim} 201 202 205 """ 203 206 Module of Supersingular Points 204 207 … … 216 219 [ 217 220 (Vector space of degree 33 and dimension 1 over Finite Field of size 97 218 221 Basis matrix: 219 [ 0 0 0 1 96 96 1 0 95 1 1 1 1 95 2 96 0 0 96 96 0 0 96 220 2 96 96 2 0 0 1 1 95 0], 1), 222 [ 0 0 0 1 96 96 1 96 96 0 2 96 96 0 1 0 1 2 95 0 1 1 0 1 0 95 0 96 95 1 96 0 2], True), 221 223 (Vector space of degree 33 and dimension 1 over Finite Field of size 97 222 224 Basis matrix: 223 [ 0 1 96 16 75 22 81 0 0 17 17 80 80 0 0 16 1 40 74 23 57 96 81 224 0 23 74 0 0 0 24 73 0 0], 1), 225 [ 0 1 96 75 16 81 22 17 17 0 0 80 80 1 16 40 74 0 0 96 81 23 57 74 0 0 0 24 0 23 73 0 0], True), 225 226 (Vector space of degree 33 and dimension 1 over Finite Field of size 97 226 227 Basis matrix: 227 [ 0 1 96 90 90 7 7 0 0 6 91 6 91 0 0 7 13 0 91 6 0 84 90 228 0 91 6 0 0 0 90 7 0 0], 1) 228 [ 0 1 96 90 90 7 7 6 91 0 0 91 6 13 7 0 91 0 0 84 90 6 0 6 0 0 0 90 0 91 7 0 0], True) 229 229 ] 230 230 sage: len(D) 231 231 9 232 232 233 233 We compute a Hecker operator on a space of huge dimension! 234 234 sage: X = SupersingularModule(next_prime(100000)) 235 sage: t = X.T(2).matrix() # long time (but still less than a minute!)236 sage: t.nrows() # long time235 sage: t = X.T(2).matrix() # long time (but still less than a minute!) 236 sage: t.nrows() # long time 237 237 8334 238 239 TESTS: 240 sage: X = SupersingularModule(389) 241 sage: T = X.T(2).matrix().change_ring(QQ) 242 sage: d = T.decomposition() 243 sage: len(d) 244 6 245 sage: [a[0].dimension() for a in d] 246 [1, 1, 2, 3, 6, 20] 247 sage: loads(dumps(X)) == X 248 True 249 sage: loads(dumps(d)) == d 250 True 238 251 """ 239 252 240 253 #***************************************************************************** … … 248 261 }%small 249 262 250 263 All code included with \sage must be licensed under the GPL or a less 251 restrictive license (e.g., the BSD license). You should share252 copyright with William Stein by including your name at the top.It is {\em very264 restrictive license (e.g., the BSD license). 265 It is {\em very 253 266 important} that you include your name in the AUTHOR log, since 254 267 everybody who submits code to \sage to receive proper credit. (If 255 268 ever you feel you are not receiving proper credit for anything you 256 submit to \sage, please let William Steinknow!)269 submit to \sage, please let the development team know!) 257 270 258 271 \section{Documentation Strings}\label{sec:conv-docs} 259 {\bf Every} function should have a docstring that includes 272 273 \textbf{Every} function must have a docstring that includes 260 274 the following information: 261 275 \begin{itemize} 262 276 \item A one-sentence description of the function, followed by a blank line. … … 268 282 \item An EXAMPLES block for examples. This is {\em not} optional. 269 283 \item A NOTES block for special notes (optional). Include information 270 284 such as algorithm used, references, etc. 271 \item An AUTHORS block (important!). 285 \item An AUTHORS block (optional, but encouraged for important 286 functions, so users can see from the docstring who wrote it and 287 therefore whom to contact if they have questions). 272 288 \end{itemize} 273 289 Use the following template when documenting functions. Note the 274 290 indentation: 275 291 %skip 276 292 \begin{verbatim} 277 293 def point(self, x=1, y=2): 278 r""" # note the r for "raw string"294 r""" 279 295 This function returns the point $(x^5,y)$. 280 296 281 297 INPUT: … … 321 337 \begin{itemize} 322 338 323 339 \item Use nice \Latex formating everywhere but in the 324 \code{INPUT/OUTPUT} blocks. 340 \code{INPUT/OUTPUT} blocks. If you use backslashes, either use 341 double backslashes or place an {\tt r} right before the first triple 342 opening quote. For example, 343 %skip 344 \begin{verbatim} 345 def cos(x): 346 """ 347 Returns $\\cos(x)$. 348 """ 349 350 def sin(x): 351 r""" 352 Returns $\sin(x)$. 353 """ 354 \end{verbatim} 355 356 Also, the non-standard \Latex macros that are used when typesetting the 357 documentation are defined in 358 \code{SAGE_ROOT/doc/commontex/macros.tex}. If you need to add more 359 macros, add them there and post a patch file on the \sage trac server 360 (Chapter~\ref{ch:trac}) so that they can be included in the next 361 version of \sage. 325 362 326 363 \item Liberally describe what the examples do. Note that there must 327 364 be a blank line after the example code and before the explanatory … … 334 371 regular basis, and are crucial for the quality and adaptability of 335 372 \sage. Without such examples, small changes to one part of \sage 336 373 that break something else might not go seen until much later when 337 someone uses the system, which is frustrating.374 someone uses the system, which is unacceptable. 338 375 \end{itemize} 339 376 340 377 The code in the examples should pass automatic testing. This means 341 378 that if the above code is in the file \code{f.py} (or \code{f.sage}), 342 379 then \code{sage -t f.py} should not give any error messages. Testing 343 380 occurs with full \sage preparsing of input within the standard \sage 344 shell environment . {\em Important.}\todo{Clarify with an345 example/pathology and explanation. }The file \code{f.py} is not381 shell environment, as described in Section~\ref{sec:Preparsing}. 382 {\em Important:} The file \code{f.py} is not 346 383 imported when running tests unless you have arranged that it be 347 384 imported into your \sage environment, i.e., unless its functions are 348 available when you start \sage using the \code{sage} command. 385 available when you start \sage using the \code{sage} command. For 386 example, the function \code{cdd_convert} in the file 387 \code{SAGE_ROOT/devel/sage/sage/geometry/polyhedra.py} includes 388 an EXAMPLES block containing the following: 389 \begin{verbatim} 390 sage: from sage.geometry.polyhedra import cdd_convert 391 sage: cdd_convert(' 1 1 0 0') 392 [1, 1, 0, 0] 393 \end{verbatim} 394 \sage doesn't know about the function \code{cdd_convert} by default, 395 so it needs to be imported before it is tested; hence the first line 396 in the example. 349 397 350 \subsection{Further conventions for automated testing of examples} 398 399 400 \subsection{Further Conventions for Automated Testing of Examples} 351 401 \label{sec:Further-conventions} 352 402 The Python script \code{local/bin/sage-doctest} implements 353 documentation testing in \sage (see Chapter~\ref{ch:testing} for more403 documentation testing in \sage (see Section~\ref{ch:testing} for more 354 404 details). When writing documentation, keep the following points in 355 405 mind: 356 406 \begin{itemize} … … 359 409 \code{2/3} as a rational instead of the Python int \code{0}. For 360 410 more information on preparsing see Section \ref{sec:Preparsing}. 361 411 362 \item If a test line contains the text \code{random} it is executed by412 \item If a test line contains the text \code{random}, it is executed by 363 413 \code{sage-doctest} but \code{sage-doctest} does not check that the 364 414 output agrees with the output in the documentation string. 415 For example, the docstring for the \code{__hash__} method for 416 \code{CombinatorialObject} in 417 \code{SAGE_ROOT/devel/sage/sage/combinat/combinat.py} includes the 418 lines 419 %skip 420 \begin{verbatim} 421 sage: hash(c) #random 422 1335416675971793195 423 sage: c._hash #random 424 1335416675971793195 425 \end{verbatim} 365 426 366 427 \item If a line contains the text \code{long time} then that line is 367 428 not tested unless the {\code -long} option is given, e.g., … … 370 431 \sage development, but will get run before major releases. No 371 432 example should take more than about 30 seconds. 372 433 373 Here is the \code{Lseries} function docstring which contains the phrase374 \code{long time} and is taken from375 \code{SAGE_ROOT/devel/sage/sage/schemes/elliptic_curves/ell_rational.py} 376 434 For example, here is part of the docstring from the \code{regulator} method for 435 rational elliptic curves, from the file 436 \code{SAGE_ROOT/devel/sage/sage/schemes/elliptic_curves/ell_rational.py}: 437 %skip 377 438 \begin{verbatim} 378 def Lseries(self, s): 379 r""" 380 Returns the value of the L-series of the elliptic curve E at s, where s 381 must be a real number. 382 383 Use self.Lseries_extended for s complex. 384 385 \note{If the conductor of the curve is large, say $>10^{12}$, 386 then this function will take a very long time, since it uses 387 an $O(\sqrt{N})$ algorithm.} 388 389 EXAMPLES: 390 sage: E = EllipticCurve([1,2,3,4,5]) 391 sage: E.Lseries(1) 392 0.000000000000000 393 sage: E.Lseries('1.1') # long time (!) 394 0.28549100767814833 439 sage: E = EllipticCurve([0, 0, 1, -1, 0]) 440 sage: E.regulator() # long time (1 second) 441 0.0511114082399688 395 442 \end{verbatim} 396 397 443 398 444 \item If a line contains \code{todo: not implemented}, it is never 399 445 tested. It is good to include lines like this to make clear … … 409 455 \code{sage -t -optional f.py}. (Note that \code{-optional} 410 456 must not be the first argument to \code{sage}.) Use this 411 457 to include doctests that require optional packages. 412 For example, 458 For example, the docstring for \code{_magma_init_} in the 459 class \code{EllipticCurve_finite_field} from the 460 \code{SAGE_ROOT/devel/sage/sage/schemes/elliptic_curves/ell_finite_field.py} 461 contains 413 462 \begin{verbatim} 414 sage: E .padic_regulator(5) # requires optional MAGMA package415 xxx463 sage: EllipticCurve(GF(41),[2,5])._magma_init_() # optional -- requires Magma 464 'EllipticCurve([_sage_[1]|GF(41)!0,GF(41)!0,GF(41)!0,GF(41)!2,GF(41)!5])' 416 465 \end{verbatim} 417 466 418 467 \item If the entire documentation string contains all three words … … 424 473 425 474 \end{itemize} 426 475 427 Using \code{sage_search} from the \sage prompt or \code{grep} one can 428 easily find aforementioned keywords and in the case of \code{todo: not 429 implemented} use the results to motivate further development on \sage. 476 Using \code{search_src} from the \sage prompt (or \code{grep}), one can 477 easily find the aforementioned keywords. In the case of \code{todo: not 478 implemented}, one can use the results of such a search to direct 479 further development on \sage. 430 480 431 \section{Exceptions} 432 Whenever code such as the following should be avoided: 481 482 \section{Automated Testing}\label{ch:testing} 483 484 This section describes \sage's automated testing of test files of the 485 following types: \code{.py .pyx .sage .tex}. Briefly, use \code{sage -t 486 <file>} to test that the examples in \code{<file>} behave exactly as 487 claimed. See the following subsections for more details. 488 See also Section~\ref{sec:conv-docs} for a discussion of how to 489 include examples in documentation strings and what conventions to 490 follow. 491 492 \subsection{Testing .py, .pyx and .sage Files}\label{sec:test:python} 493 494 Run \code{sage -t <filename.py>} to test that all code examples in 495 \code{filename.py}. Similar remarks apply to \code{.sage} and 496 \code{.pyx} files. 497 498 \begin{verbatim} 499 sage -t [--verbose] [--optional] [files and directories ... ] 500 \end{verbatim} 501 502 When you run \code{sage -t <filename.py>}, \sage makes a copy of 503 \code{<filename.py>} with all the \code{sage} prompts replaced by 504 \code{{>}{>}>}, then uses the standard Python doctest framework to test 505 the documentation. More precisely, 506 the Python script \code{local/bin/sage-doctest} implements 507 documentation testing. It does the following when asked 508 to test a file \code{foo.py} or \code{foo.sage}. 509 \begin{enumerate} 510 511 \item Creates the directory \code{.doctest} if it doesn't 512 exist and the file \code{.doctest/foo.py}. 513 514 \item The file \code{.doctest_foo.py} contains functions for 515 each docstring in \code{foo.py}, but with all \sage preparsing 516 applied and with \code{from sage.all import *} at the top. 517 The function documentation is thus standard Python with 518 \code{{>}{>}>} prompts. 519 520 \item \code{sage-doctest} then runs \sage's Python interpreter 521 on \code{.doctest_foo.py}. 522 \end{enumerate} 523 524 \note{{\em Note that line numbers in the errors 525 you might see apply to the file \code{.doctest_foo.py}, not 526 to the original file \code{foo.py}!} If you get errors, 527 feel free to inspect \code{.doctest_foo.py}; it is never 528 automatically deleted by \sage.} 529 530 Your file passes these tests if the code in it will run when entered 531 at the \code{sage:} prompt with no special imports. Thus users are 532 guaranteed to be able to exactly copy code out of the examples you 533 write for the documentation and have them work. 534 535 536 537 \subsection{Testing \Latex Documentation} 538 539 Run \code{sage -t <filename.tex>} to test the examples in 540 verbatim environments in \Latex documentation. Put 541 \code{\%skip} 542 on the line right before a verbatim environment to have that 543 example skipped when testing the file. 544 545 \sage creates a file \code{.doctest_filename.py} and tests 546 it just as for \code{.py}, \code{.pyx} and \code{.sage} 547 files. In order to skip testing of a block of code 548 in a verbatim environment, put the \Latex comment 549 \code{\%skip} on the previous line. 550 551 Of course in \Latex files, one 552 often inserts explanatory texts between different 553 verbatim environments. To link together verbatim 554 environments use the \code{\%link} comment. For 555 example: 433 556 %skip 434 \begin{verbatim} 435 try: 436 some code 437 except: # bad 438 more code 439 \end{verbatim} 440 Instead catch specific exceptions. For example, 441 \begin{verbatim} 442 try: 443 return self.__coordinate_ring 444 except (AttributeError, other exceptions), msg: # Good 445 more code to compute something 446 \end{verbatim} 447 Note that the syntax in the except is to list all the exceptions 448 that are caught as a tuple, followed by the variable that holds 449 the message. 557 \begin{newverbatim} 558 \begin{verbatim} 559 sage: a = 1 560 \end{verbatim}%link 450 561 451 If you don't have any exceptions explicitly listed (as a tuple), your 452 code will catch absolutely anything, including ctrl-C and alarms, 453 and this will lead to confusion. 562 Next we add 1 to \code{a}. 563 %link 564 \begin{verbatim} 565 sage: 1 + a 566 2 567 \end{verbatim} 568 \end{newverbatim} 454 569 455 \section{Optional Packages} If an optional package is required for a 456 certain function to work, that function should fail gracefully --- 457 perhaps using a \code{try} and \code{except} block --- when the optional 458 package is not available, and should give a hint about how to install 459 it. For example, typing \code{sage -optional} gives a list of all 460 optional packages, so suggest to the user that they type that. 570 See \code{SAGE_ROOT/doc/tut/tut.tex} for many examples 571 of how to include automated testing in \Latex documentation 572 for \sage. 573 574 575 576 \section{Randomized Testing} 577 \label{ch:randomtesting} 578 579 In addition to all the examples in your docstrings, which serve as 580 both demonstrations and tests of your code, you should consider creating a 581 test suite. Think of this as a program that will run for a while and 582 ``tries'' to crash your code using randomly generated input. Your 583 test code should define a class \class{Test} with a {\tt random()} 584 method that runs random tests. These are all assembled together 585 later, and each test is run for a certain amount of time on a regular 586 basis. 587 588 For example, see the file \code{SAGE_ROOT/devel/sage/sage/modular/modsym/tests.py}. 589 590 591 592 \chapter{Coding in Python for \SAGE} 593 594 This chapter discusses some issues with, and advice for, coding in \SAGE. 595 596 597 \section{Design} 598 599 If you are planning to develop some new code for \sage, design is 600 important. So think about what your program will do and how that fits 601 into the structure of \SAGE. In particular, much of \SAGE is 602 implemented in the object-oriented language Python, and there is a 603 hierarchy of classes that organize code and functionality. For 604 example, if you implement elements of a ring your class should derive 605 from \class{sage.structure.element.RingElement}, rather than starting 606 from scratch. Try to figure out how your code should fit in with 607 other \sage code, and design it accordingly. 608 461 609 462 610 \section{Special \sage Functions} 463 611 … … 472 620 also has special methods. They are typically of the form 473 621 \code{_XXX_}. (In a few cases the trailing underscore is not 474 622 included, but this will be changed so that the trailing underscore is 475 always included.) This section describes allspecial methods.623 always included.) This section describes these special methods. 476 624 477 625 All objects in \sage should derive from the Cython extension 478 626 class \code{SageObject}: … … 483 631 class MyClass(SageObject,...): 484 632 ... 485 633 \end{verbatim} 634 or from some other already existing \sage class: 635 %skip 636 \begin{verbatim} 637 from sage.rings.ring import Algebra 638 639 class MyFavoriteAlgebra(Algebra): 640 ... 641 \end{verbatim} 486 642 487 643 You should implement the \code{\_latex\_} and \code{\_repr\_} method 488 644 for every object. The other methods depend on the nature of the … … 491 647 \subsection{\Latex Representation} 492 648 Every object x in \sage should support the command \code{latex(x)}, so 493 649 that any \sage object can be easily and accurately displayed via 494 \Latex. Here is how to make a class and therefore its instances650 \Latex. Here is how to make a class (and therefore its instances) 495 651 support the command \code{latex}. 496 652 497 653 \begin{enumerate} 498 654 499 \item Put \code{import sage.misc.latex as latex} at the top of the 500 file that defines your class. 501 502 \item Define a member function \code{_latex_(self)} that returns a 655 \item Define a method \code{_latex_(self)} that returns a 503 656 \Latex representation of your object. It should be something that 504 657 can be typeset correctly within math mode. 505 658 506 \item Often objects are built up out of other options. For example if 659 \item Often objects are built up out of other \sage objects, and these 660 components should be typeset using the \code{latex} function. 661 For example if 507 662 \code{c} is a coefficient of your object, and you want to typeset 508 663 \code{c} using latex, use \code{latex(c)} instead of 509 664 \code{c._latex_()}, since \code{c} might not have a \code{_latex_} … … 513 668 illustrates latex generation for your object. 514 669 515 670 \item You can use any macros included in \code{amsmath}, 516 \code{amssymb}, \code{amsfonts}or the ones defined in671 \code{amssymb}, or \code{amsfonts}, or the ones defined in 517 672 \code{SAGE_ROOT/doc/commontex/macros.tex}. 518 519 \item Use \code{view(x)} to view the typeset version of an object x.520 673 521 674 \end{enumerate} 522 675 %skip 523 676 524 An example template for \Latex representationfollows:677 An example template for a \code{_latex_} method follows: 525 678 526 679 \begin{verbatim} 527 import sage.misc.latex as latex528 ...529 680 class X: 530 681 ... 531 682 def _latex_(self): … … 540 691 return `\\frac{%s}{%s}''%(latex(self.numer), latex(self.denom)) 541 692 \end{verbatim} 542 693 694 As shown in the example, \code{latex(a)} will produce \Latex code 695 representing the object \code{a}. Calling \code{view(a)} will display the 696 typeset version of this. 697 698 699 543 700 \subsection{Print Representation} 544 701 The standard Python printing method is \code{__repr__(self)}. In 545 702 \sage, that is for objects that derive from \code{SageObject} (which 546 is everything in \sage), instead define \code{_repr_(self)}. This is547 preferable because if you only define \code{_repr_(self)} and not 548 \code{__repr__(self)}, then users can rename your object to print549 however they like.703 is everything in \sage), instead define \code{_repr_(self)}. 704 % This is preferable because if you only define \code{_repr_(self)} and 705 % not \code{__repr__(self)}, then users can rename your object to print 706 % however they like. 550 707 551 708 Here is an example of the \code{_latex_} and \code{_repr_} functions 552 709 for the Pi class. It is from the file 553 \code{SAGE_ROOT/devel/sage/sage/functions/constants.py} 710 \code{SAGE_ROOT/devel/sage/sage/functions/constants.py}: 554 711 555 \begin{verbatim}%skip 712 %skip 713 \begin{verbatim} 556 714 class Pi(Constant): 557 715 """ 558 716 The ratio of a circle's circumference to its diameter. … … 571 729 return "\\pi" 572 730 \end{verbatim} 573 731 574 \subsection{Matrix from Object}575 Provide a \code{_matrix_} me mber functionfor an object that can be732 \subsection{Matrix or Vector from Object} 733 Provide a \code{_matrix_} method for an object that can be 576 734 coerced to a matrix over a ring $R$. Then the \sage function 577 735 \code{matrix} will work for this object. 578 736 579 The following is from \code{SAGE_ROOT/devel/sage/sage/graphs/graph.py} 737 The following is from \code{SAGE_ROOT/devel/sage/sage/graphs/graph.py}: 580 738 581 739 \begin{verbatim} 582 class SimpleGraph(GenericGraph): 583 ... 584 def _matrix_(self, R): 585 return self.am() 740 class GenericGraph(SageObject): 741 ... 742 def _matrix_(self, R=None): 743 if R is None: 744 return self.am() 745 else: 746 return self.am().change_ring(R) 586 747 587 def am(self): 588 """589 Shorter call for adjacency matrix makes life easier.590 """ 591 return self.adjacency_matrix()748 749 def adjacency_matrix(self, sparse=None, boundary_first=False): 750 ... 751 752 am = adjacency_matrix # shorter call makes life easier 592 753 \end{verbatim} 593 754 594 \subsection{Vector from Object} 595 Provide a \code{_vector_} member functionfor an object that can be755 Similarly, 756 provide a \code{_vector_} method for an object that can be 596 757 coerced to a vector over a ring $R$. Then the \sage function 597 \code{vector} will bework for this object.758 \code{vector} will work for this object. 598 759 599 760 \todo{Provide example from a .py file 600 761 601 762 The following is from the file 602 SAGE_ROOT/sage/sage/modules/free_module_element.pyx763 \code{SAGE_ROOT/sage/sage/modules/free_module_element.pyx}: 603 764 604 765 \begin{verbatim} 605 766 cdef class FreeModuleElement(element_Vector): # abstract base class … … 610 771 } 611 772 612 773 774 \section{\SAGE Preparsing} 775 \label{sec:Preparsing} 776 777 The following files are relevant to preparsing in \sage: 778 779 \begin{enumerate} 780 781 \item \code{SAGE_ROOT/local/bin/sage-sage}, 782 783 \item \code{SAGE_ROOT/local/bin/sage-preparse}, 784 785 \item \code{SAGE_ROOT/devel/sage/sage/misc/preparser.py} 786 787 % \item \code{SAGE_ROOT/devel/sage/sage/misc/preparser_ipython.py} 788 789 \todo{Talk about 790 \code{SAGE_ROOT/devel/sage/sage/misc/preparser_ipython.py} file} 791 792 \end{enumerate} 793 794 In particular, the file \code{preparser.py} contains the \SAGE 795 preparser code. Here are some , and the following are some notes from it: 796 797 \begin{itemize} 798 799 \item In \SAGE, methods can be called on integer and real literals -- 800 note that in pure Python this would be a syntax error. For example: 801 \begin{verbatim} 802 sage: 16.sqrt() 803 4 804 sage: 87.factor() 805 3 * 29 806 \end{verbatim} 807 808 \item Raw literals are not preparsed, which can be useful from an 809 efficiency point of view. Just like Python ints are denoted by an 810 L, in \SAGE raw integer and floating literals are followed by an ``r'' 811 (or ``R'') for raw, meaning not preparsed. Example: 812 \begin{verbatim} 813 sage: a = 393939r 814 sage: a 815 393939 816 sage: type(a) 817 <type 'int'> 818 sage: b = 393939 819 sage: type(b) 820 <type 'sage.rings.integer.Integer'> 821 sage: a == b 822 True 823 \end{verbatim} 824 825 \item Raw literals can be very useful in certain cases. For instance, 826 Python integers are typically much more efficient than \SAGE integers 827 when they are very small; large \SAGE integers are much more 828 efficient than Python integers, since they are implemented using the 829 GMP C library. For example, the first of these commands may be somewhat 830 slower than the second: 831 \begin{verbatim} 832 sage: v = [ 2*i for i in range(10000)] 833 sage: v = [ 2r*i for i in range(10000r)] 834 \end{verbatim} 835 because the first uses \sage integers, while the second uses raw 836 Python integers. 837 838 \end{itemize} 839 840 Consult the file \code{preparser.py} for more details about \SAGE 841 preparsing, more examples involving raw literals, etc. 842 843 When a file \code{foo.sage} is loaded in a \sage session, a preparsed version of 844 \code{foo.sage} is created titled \code{foo.py}. The beginning of \code{foo.py} states: 845 \verb+This file was *autogenerated* from the file foo.sage.+ 846 847 613 848 \section{The \SAGE Rules and Conventions for Coercion and Arithmetic} 614 \note{These are the {\bf planned} rules and conventions. They have not615 been systematically implemented throughout the system yet.}616 849 617 Let $R$ and $S$ be \SAGE parent structures, e.g., 618 groups, rings, point sets, etc.\todo{Make section gentler} 850 \note{August 2008: Much of this material is out of date. We are 851 working on a revised version.} 852 853 \todo{Make section gentler} 854 855 Let $R$ and $S$ be \SAGE parent structures, e.g., groups, rings, point 856 sets, etc. 619 857 620 858 \begin{itemize} 621 \item {\bfOBJECT COERCION \code{_coerce_}:}859 \item \textbf{OBJECT COERCION \code{_coerce_}:} 622 860 Suppose a \code{_coerce_} map $R \to S$ is defined. This 623 861 is a method of $S$, so it is called using the notation \code{S._coerce_(x)}, where $R$ is a parent of $x$. 624 862 It's signature is … … 679 917 rules. E.g., embeddings of finite fields via Conway polynomials, or 680 918 inclusions of extensions of number fields, fit into this structure, as 681 919 does the inclusion $\QQ \hookrightarrow \CC$ and the surjection $\ZZ 682 \to \ZZ/n\ZZ$. The function \code{_coerce_} {\bfdoes not} have to be920 \to \ZZ/n\ZZ$. The function \code{_coerce_} \textbf{does not} have to be 683 921 ``canonical'' in a precisely defined mathematical sense. 684 922 685 \item {\bfOBJECT CREATION \code{__call__}:}923 \item \textbf{OBJECT CREATION \code{__call__}:} 686 924 If you write $R(x)$, then the \code{__call__} method of $x$ is called. It 687 925 should make an object of $R$ from $x$, if at all possible. 688 926 \code{__call__} is never called implicitly by binary operators. 689 927 If $x$ already has $R$ as its parent, then \code{__call__} must return 690 a {\bfnew copy} of $x$, unless $x$ is immutable (this is to agree with928 a \textbf{new copy} of $x$, unless $x$ is immutable (this is to agree with 691 929 standard Python conventions). 692 930 For example, if $x \in \ZZ/n\ZZ$, then \code{ZZ.__call__(x)} is defined but 693 931 \code{ZZ.__coerce__(x)} is not. 694 932 695 \item {\bfARITHMETIC \code{__add__}, \code{__mul__}, ...:}: When933 \item \textbf{ARITHMETIC \code{__add__}, \code{__mul__}, ...:}: When 696 934 doing a binary operation, if the parents are not identical (in the 697 935 sense of is\todo{incomplete?}), determine if precisely one {\tt 698 936 \_coerce\_} map is defined; if so, apply it and do the arithmetic 699 937 operation. If both are defined, the parents are canonically 700 938 isomorphic, so use the left one. If neither are defined, raise a 701 TypeError. (Whether or not there is a coerce map between objects \todo{the objects?}939 TypeError. (Whether or not there is a coerce map between objects 702 940 should be cached for efficiency.) 703 941 704 \item {\bfPARENT OBJECT \code{__cmp__}}: \code{R == S} by definition942 \item \textbf{PARENT OBJECT \code{__cmp__}}: \code{R == S} by definition 705 943 means that \code{_coerce_} is defined in both directions. Roughly 706 944 speaking this means that $R$ and $S$ are identical or canonically 707 945 isomorphic, though there could be some exceptions to this. Note … … 709 947 considered equal, since there are never canonical coercion maps in 710 948 both directions when the categories are different. 711 949 712 \item {\bfELEMENT \code{__cmp__}}:950 \item \textbf{ELEMENT \code{__cmp__}}: 713 951 If the parents aren't identical, test if 714 precisely one \code{_coerce_} map is defined -- if so, return \code{__cmp__}952 precisely one \code{_coerce_} map is defined --- if so, return \code{__cmp__} 715 953 after applying the coerce. If both coercions are defined, compute 716 954 both \code{__cmp__}'s (in both $R$ and $S$). Return the value if they give 717 955 the same results. Otherwise return $-1$ (the convention in Python 718 956 is that \code{__cmp__} never raises an exception). If no \code{_coerce_} is 719 957 defined return $-1$ (i.e., not equal). 720 958 721 \item {\bfIN \code{__contains__}}:959 \item \textbf{IN \code{__contains__}}: 722 960 \code{x in R} should be true if and only if \code{R._coerce_(x)} would not 723 961 raise a \code{TypeError}. 724 962 \end{itemize} 725 963 964 726 965 \section{Mutability} 966 727 967 Parent structures (e.g., rings, fields, matrix spaces, etc.) should 728 968 be immutable and globally unique whenever possible. Immutability 729 includes making it soproperties like generator labels and default969 means, among other things, that properties like generator labels and default 730 970 coercion precision cannot be changed. 731 971 732 Global uniqueness while not wasting space storing objects that are not733 being referencedis best implemented using the standard Python weakref972 Global uniqueness while not wasting memory 973 is best implemented using the standard Python weakref 734 974 module, a factory function, and module scope 735 variable.\todo{Rewrite. Difficult to parse. Make gentler} 975 variable. 976 \todo{Rewrite. Difficult to parse. Make gentler} 736 977 737 \todo{Put a tutorial on this here} .978 \todo{Put a tutorial on this here} 738 979 739 980 Certain objects, e.g., matrices, may start out mutable and become 740 981 immutable later. See the file \code{SAGE_ROOT/devel/sage/sage/structure/mutability.py}. 741 982 742 \section{Importing Modules}743 Do not import at the top level of your module a third-party module that744 will take a long time to initialize (e.g., matplotlib). It is important745 that \code{from sage.all import *} not take forever, since this is what746 dominates the \SAGE startup time.747 748 On a somewhat regular basis I do the following:749 \begin{verbatim}750 echo "%prun import sage.all" | sage -ipython|more751 \end{verbatim}752 in order to profile what is taking a long time during753 the load. In many cases I use the locate command to754 find files mentioned in the profile.755 983 756 984 \section{The {\tt \_\_hash\_\_} Special Method} 985 757 986 Here's the definition of \code{__hash__} 758 987 from the Python reference manual. 759 988 … … 780 1009 Python language, {\em which in \sage we simply cannot make} (!), and 781 1010 violating it has consequences. Fortunately, the consequences are pretty 782 1011 clearly defined and reasonably easy to understand, so if you know 783 about them they don't cause you trouble. I think the following1012 about them they don't cause you trouble. The following 784 1013 example illustrates them pretty well: 785 1014 \begin{verbatim} 786 1015 sage: v = [Mod(2,7)] … … 800 1029 KeyError: 9 801 1030 \end{verbatim} 802 1031 803 Here's another example ,1032 Here's another example: 804 1033 \begin{verbatim} 805 1034 sage: R = RealField(10000) 806 1035 sage: a = R(1) + R(10)^-100 … … 811 1040 812 1041 Unfortunately, in \SAGE we simply can't require: 813 1042 \begin{verbatim} 814 (*)"a == b ==> hash(a) == hash(b)"1043 (*) "a == b ==> hash(a) == hash(b)" 815 1044 \end{verbatim} 816 1045 because serious mathematics is simply too complicated for this 817 1046 rule. For example, the equalities … … 820 1049 integers. 821 1050 822 1051 The only way we could ``fix'' this problem for good would be to 823 abandon using the "==" operator for "\SAGE equality", and implement1052 abandon using the \code{==} operator for ``\SAGE equality'', and implement 824 1053 \SAGE equality as a new method attached to each object. Then we could 825 follow Python rules for "=="and our rules for everything else, and1054 follow Python rules for \code{==} and our rules for everything else, and 826 1055 all \SAGE code would become completely unreadable (and for that matter 827 1056 unwritable). So we just have to live with it. 828 1057 829 So what is done in \sage is to {\em attempt} to satisfy (*)when it1058 So what is done in \sage is to {\em attempt} to satisfy \verb+(*)+ when it 830 1059 is reasonably easy to do so, but use judgment and not go overboard. 831 1060 For example, 832 1061 \begin{verbatim} 833 1062 sage: hash(Mod(2,7)) 834 1063 2 835 1064 \end{verbatim} 836 I.e., we get 2. That's better than some weirdrandom hash that also1065 The output 2 is better than some random hash that also 837 1066 involves the moduli, but it's of course not right from the Python point 838 1067 of view, since \code{9 == Mod(2,7)}. 839 1068 840 1069 The goal is to make a hash function that is fast but within reason 841 respects any obvious ,natural inclusions and coercions.1070 respects any obvious natural inclusions and coercions. 842 1071 843 1072 1073 \section{Exceptions} 844 1074 845 846 847 \chapter{Creating a \SAGE Module or Package} 848 849 \todo{What is a \SAGE Module and what is a \SAGE Package? Are they the 850 same? This chapter talks about only the latter? When 851 should/shouldn't one make a package out of a bunch of files?} 852 853 \begin{enumerate} 854 \item Design: Think about what your program will do and how that fits into the 855 structure of \SAGE. In particular, much of \SAGE is implemented in 856 the object-oriented language Python, and there is a hierarchy of 857 classes that organize code and functionality. It is {\em 858 surprisingly painful and difficult} to create some Python classes, 859 then realize that much functionality should be factored out into 860 base classes, and to do it\todo{Not sure what you are trying to 861 convey here}. So try to get the structure right the first time. 862 For example, if you implement elements of a ring your class should 863 derive from 864 \class{sage.structure.element.RingElement}.\todo{Design is being 865 talked about without using the word. Consciously avoiding the 866 term?} 867 868 \item Coding: Code it up, following the conventions in 869 Chapter~\ref{ch:conventions}. Make the basic structure and 870 interface for what you're doing in Python. You can use any of the 871 following languages to implement the hard parts: Python, C/C++, 872 Cython, Fortran 95, GAP, Singular, and GP/PARI. You can also use the mwrank, 873 GSL, NTL, and PARI C/C++ libraries. (And if you are OK with your code 874 depending on optional \sage packages, you can use Octave, or 875 even Magma, Mathematica, or Maple.) For readability and usability 876 it is best to use \sage when possible. On the other hand, for 877 efficiency reasons it is often crucial to use a compiled language or 878 an existing library or program---do not reinvent the wheel or write 879 code that is so slow as to be only a toy\todo{I split the 880 original line into two, as reusing existing code should be 881 emphasized more}. 882 883 \item Documentation: Document {\em every} Python function that you've 884 defined, 885 886 \item Testing: Liberally include examples in your source code. See 887 Chapter~\ref{ch:testing} for more on automated testing of examples. 888 889 \item (Optional) Write a file that generates random input to test (and 890 break!) your code with the intention of making it robust (See 891 Chapter~\ref{ch:randomtesting}). 892 893 \item (Optional) Send your package to the \code{sage-devel} Google group 894 for inclusion in \SAGE, or make a standalone 895 {\tt spkg} (see Chapter~\ref{ch:spkg}). 896 897 \end{enumerate} 898 899 900 \section{\SAGE Preparsing} 901 \label{sec:Preparsing} 902 903 The following is a list of files relevant to preparsing through which 904 control flows when \sage is run: 905 906 \begin{enumerate} 907 908 \item \code{SAGE_ROOT/sage-sage}, 909 910 \item \code{SAGE_ROOT/sage-preparse}, 911 912 \item \code{SAGE_ROOT/devel/sage/sage/misc/preparser.py} 913 914 \todo{Talk about 915 \code{SAGE_ROOT/devel/sage/sage/misc/preparser_ipython.py} file} 916 917 \end{enumerate} 918 919 The file preparser.py is contains the \SAGE preparser code and the 920 following are some notes from the aforementioned file 921 922 \begin{itemize} 923 924 \item In \SAGE methods can also be called on integer and real literals 925 (note that in pure Python this would be a syntax error). 926 927 \item Note that calling methods on int literals in pure Python is a 928 syntax error, but \SAGE allows this for \SAGE integers and reals, 929 because users frequently request it. 930 931 \item Raw literals are not preparsed, which can be useful from an 932 efficiency point of view. Just like Python ints are denoted by an 933 L, in \SAGE raw integer and floating literals are followed by an ``r'' 934 (or ``R'') for raw, meaning not preparsed. 935 936 \item Raw literals can be very useful in certain cases. For instance, 937 Python integers are typically much more efficient than \SAGE integers 938 when they are very small; large \SAGE integers are much more 939 efficient than Python integers, since they are implemented using the 940 GMP C library --- see section \ref{sec:Benchmarking}. 941 942 \end{itemize} 943 944 945 Consult preparser.py for details about \SAGE preparsing, examples involving raw literals, etc. 946 947 When foo.sage is loaded in a sage session, a preparsed version of 948 foo.sage is created titled foo.py. The beginning of foo.py states: 949 \code{This file was *autogenerated* from the file foo.sage.} 950 951 \section{Randomized Testing} 952 \label{ch:randomtesting} 953 In addition to all the examples, which serve as both demonstrations 954 and tests of your code, you should also create a test suite. Think of 955 this as a program that will run for a while and ``tries'' to crash 956 your code using randomly generated input. Your test code should 957 define a class \class{Test} with a {\tt random()} method that runs 958 random tests. These are all assembled together later, and each test 959 is run for a certain amount of time on a regular basis. 960 961 \section{Creating a New \sage spkg Package} 962 \label{ch:spkg} 963 Here's how to make your own \sage package. 964 965 \begin{itemize} 966 \item Note that a \sage package is simply a .tar.bz2 file (or a tar 967 file), but named with .spkg to discourage confusion. In particular, 968 you can type 1075 Please avoid code like this: 969 1076 %skip 970 1077 \begin{verbatim} 971 tar jxvf my_package-version.spkg 1078 try: 1079 some code 1080 except: # bad 1081 more code 972 1082 \end{verbatim} 973 to extract one and see what is inside. 1083 Instead catch specific exceptions. For example, 1084 \begin{verbatim} 1085 try: 1086 return self.__coordinate_ring 1087 except (AttributeError, other exceptions), msg: # Good 1088 more code to compute something 1089 \end{verbatim} 1090 Note that the syntax in \code{except} is to list all the exceptions 1091 that are caught as a tuple, followed by an error message. 974 1092 975 \item 976 To create your own package: 977 \begin{itemize} 978 \item[(a)] 979 Make a directory, e.g., \code{mypackage-0.1}. 980 The name of the directory should be a lower-case string with no 981 dashes followed by a dash followed by a version number. 1093 If you don't have any exceptions explicitly listed (as a tuple), your 1094 code will catch absolutely anything, including \code{ctrl-C} and alarms, 1095 and this will lead to confusion. 982 1096 983 \item[(b)]984 Put your files in that directory.985 1097 986 \item[(c)] 987 Create an executable shell script \code{mypackage-0.1/spkg-install}. 1098 \section{Importing} 988 1099 989 \item[(d)] 990 The script \code{spkg-install} is run during installation 991 of the \sage package. In this script you may 992 make the following assumptions: 993 \begin{itemize} 994 \item 995 The PATH has \code{sage} and the \sage install's \code{python} 996 location at the front. Thus the command 1100 We mention two issues with importing: circular imports and importing 1101 large third-party modules. 1102 1103 First, you must avoid circular imports. For example, suppose that the 1104 file \code{SAGE_ROOT/devel/sage/sage/algebras/steenrod_algebra.py} 1105 started with a line 997 1106 %skip 998 1107 \begin{verbatim} 999 python setup.py install 1108 from sage.sage.algebras.steenrod_algebra_bases import * 1000 1109 \end{verbatim} 1001 will run the correct version of Python with 1002 everything setup correctly. Also, e.g., 1003 running \code{gap} or \code{Singular} will run the 1004 correct versions. 1005 1006 \item 1007 The environment variable \code{SAGE_ROOT} points to 1008 the root directory of the \sage install. 1009 \item 1010 The environment variable \code{SAGE_LOCAL} points to 1011 the \code{SAGE_ROOT/local} directory of the \sage install. 1012 \item 1013 The environment variables 1014 \code{LD_LIBRARY_PATH} and \code{DYLD_LIBRARY_PATH} 1015 both have \code{SAGE_ROOT/local/lib} at the front. 1016 1017 \end{itemize} 1018 1019 \item[(e)] 1020 The \code{spkg-install} script should copy your files to the appropriate 1021 place after doing any build that is necessary. That's it! 1022 1023 \item[(f)] (Optional) Send a copy to the \code{sage-devel} Google group 1024 so it can be posted to the \SAGE web site, so anybody can automatically 1025 install it by typing \code{sage -i my_package-version.spkg}. 1026 \end{itemize} 1027 \end{itemize} 1028 1029 For example, the location of GAP is \code{SAGE_ROOT/local/lib/gap-4.4.6/}. 1030 Thus to install gap code you have to copy it there. You should check 1031 that the directory with the given version your package is created for 1032 exists in your \code{spkg-install} script, e.g., with the following code: 1110 and that the 1111 file \code{SAGE_ROOT/devel/sage/sage/algebras/steenrod_algebra_bases.py} 1112 started with a line 1033 1113 %skip 1034 1114 \begin{verbatim} 1035 if [ ! -d $SAGE_ROOT/local/lib/gap-4.4.6 ]; then 1036 echo "This package requires that GAP 4.4.6 is installed." 1037 exit 1 1038 fi 1115 from sage.sage.algebras.steenrod_algebra import SteenrodAlgebra 1116 \end{verbatim} 1117 This sets up a loop: loading one of these files requires the other, 1118 which then requires the first, etc. 1119 1120 With this set-up, running \sage will produce an error: 1121 %skip 1122 \begin{verbatim} 1123 Exception exceptions.ImportError: 'cannot import name SteenrodAlgebra' in 'sage.rings.polynomial.polynomial_element.Polynomial_generic_dense.__normalize' ignored 1124 --------------------------------------------------------------------------- 1125 ImportError Traceback (most recent call last) 1126 1127 ... 1128 ImportError: cannot import name SteenrodAlgebra 1129 \end{verbatim} 1130 Instead, you might replace the \code{import *} line at the top of the 1131 file by more specific imports where they are needed in the code 1132 -- e.g., the \code{basis} method for the class \code{SteenrodAlgebra} 1133 might look like this (omitting the documentation string): 1134 %skip 1135 \begin{verbatim} 1136 def basis(self, n): 1137 from steenrod_algebra_bases import steenrod_algebra_basis 1138 return steenrod_algebra_basis(n, basis=self._basis_name, p=self.prime) 1039 1139 \end{verbatim} 1040 1140 1041 \emph{Caveat}: Do {\em not} just 1042 copy to \code{SAGE_ROOT/local/lib/gap*/} since that will copy 1043 your package to the lib directory of the {\em old} version of GAP if 1044 GAP is upgraded.1141 Second, do not import at the top level of your module a third-party 1142 module that will take a long time to initialize (e.g., matplotlib). 1143 As above, you might instead import specific components of the module 1144 when they are needed, rather than at the top level of your file. 1045 1145 1046 External Magma code goes in 1047 \verb+SAGE_ROOT/data/extcode/magma/user+, so if you want to redistribute 1048 Magma code with \sage as a package that Magma-enabled users can use, 1049 that's where you'd put it. You'd also want to have relevant Python code 1050 to make the Magma code easily usable. 1146 It is important to try to make \code{from sage.all import *} as fast 1147 as possible, since this is what dominates the \SAGE startup time, and 1148 controlling the top-level imports helps to do this. 1051 1149 1052 {\bf Example}:1053 These instructions provide directions for creating a particular1054 \sage package. \todo{Something about this example bugs me and I'm not sure what it is.}1055 1150 1056 \begin{itemize} 1057 \item 1058 Download \code{guava2.4.tar.gz} (the tarball for GUAVA, the 1059 GAP error-correcting codes package) from 1060 \code{http://www.gap-system.org/Packages/guava.html}. 1151 \section{Using Optional Packages} 1061 1152 1062 \item 1063 Suppose you have a directory \code{sage-package-guava}. 1064 Within that, create a directory \code{guava-2.4}. 1065 Extract the tarball into that directory 1066 (so there will be a \code{guava2.4} subdirectory of 1067 \code{sage-package-guava}). 1153 If a function requires an optional package, that function should fail gracefully --- 1154 perhaps using a \code{try} and \code{except} block --- when the optional 1155 package is not available, and should give a hint about how to install 1156 it. For example, typing \code{sage -optional} gives a list of all 1157 optional packages, so it might suggest to the user that they type that. 1068 1158 1069 \item1070 Put the following script called \code{spkg-install} in1071 \code{sage-package-guava}:1072 1159 1073 %skip1074 \begin{verbatim}1075 #!/bin/sh1076 1160 1077 if [ ! -d $SAGE_ROOT/local/lib/gap-4.4.6 ]; then 1078 echo "This package requires that GAP 4.4.6 is installed." 1079 exit 1 1080 fi 1161 \chapter{Coding in Other Languages} 1081 1162 1082 cp -pr guava2.4 $SAGE_ROOT/local/lib/gap-4.4.6/pkg/ 1163 When writing code for \sage, use Python for the basic structure and 1164 interface. For speed, efficiency, or convenience, you can implement 1165 parts of the code using any of the following languages: 1166 Cython, C/C++, Fortran 95, GAP, Singular, and GP/PARI. You can also 1167 use the mwrank, GSL, NTL, and PARI C/C++ libraries. (And if you are 1168 okay with your code depending on optional \sage packages, you can use 1169 Octave, or even Magma, Mathematica, or Maple.) 1083 1170 1084 cd $SAGE_ROOT/local/lib/gap-4.4.6/pkg/guava2.4 1085 ./configure ../.. 1086 make 1087 \ end{verbatim}1171 The first section of this chapter discusses Cython (which is a 1172 compiled language, based on Python); many components of \sage are 1173 written in Cython. Later sections discuss the interfaces between 1174 \sage and PARI, GAP, and Singular. 1088 1175 1089 Make it executable with \code{chmod +x spkg-install}.1090 1176 1091 \item1092 In the directory \code{sagefiles},1093 issue the command1094 1177 1095 %skip 1096 \begin{verbatim} 1097 <path>sage -pkg guava-2.4 1098 \end{verbatim} 1099 where \verb+<path>+ means the absolute path to the sage 1100 script. This will do some checks, run tar/bz2, and create the spkg file, 1101 \code{guava-2.4.spkg}, in the 1102 directory \code{sagefiles}, 1103 \end{itemize} 1178 \section{Cython} 1104 1179 1105 To test this, within the \code{sagefiles} directory 1106 (or whatever directory contains \code{guava-2.4.spkg}), 1107 type 1180 Cython is a compiled version of Python; it is based on Pyrex 1181 (\url{http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/}). To 1182 a large degree, Cython has changed based on what \sage's developers 1183 needed: Cython has been developed in concert with \sage. 1108 1184 1109 %skip 1110 \begin{verbatim} 1111 <path>sage -i guava-2.4.spkg 1112 \end{verbatim} 1113 It should install and compile the guava program in sage. 1114 To test its functionality, one could do the following: 1185 As such, it is a young, but developing, language, with young, but 1186 developing, documentation. See its web page, 1187 \url{http://www.cython.org/}, for the most up-to-date information. 1115 1188 1116 \begin{verbatim} 1117 sage: gap(`LoadPackage("guava")') # requires optional package 1118 true 1119 sage: gap(`HammingCode(3,3)') # requires optional package 1120 a linear [13,10,3]1 Hamming (3,3) code over GF(3) 1121 \end{verbatim} 1122 1123 To build again if you've already installed the package, use 1124 %skip 1125 \begin{verbatim} 1126 <path>sage -f guava-2.4.spkg 1127 \end{verbatim} 1128 1129 1130 \chapter{Documentation} 1131 1132 \section{Source Code Documentation} 1133 Use \Latex formatting in the documentation string. If you wish to use 1134 backslashes in the documentation string, use double backslashes or place an {\tt r} right before the first triple opening quote. For example, 1135 %skip 1136 \begin{verbatim} 1137 def cos(x): 1138 """ 1139 Returns $\\cos(x)$. 1140 """ 1141 1142 def sin(x): 1143 r""" 1144 Returns $\sin(x)$. 1145 """ 1146 \end{verbatim} 1147 1148 The non-standard macros that are used when typesetting the 1149 documentation are defined in 1150 \code{SAGE_ROOT/doc/commontex/macros.tex}. If you need to add more macros, 1151 add them there and post them on the \code{sage-devel} Google group so that 1152 they can be included in the next version of \sage. 1153 1154 1155 The first line two or three lines of the docstring should briefly summarize 1156 what the function does, e.g., ``Returns X'' or ``Computes X'', 1157 or ``Creates a new X''. Next describe the input and output of 1158 the function using the following notation: 1159 %skip 1160 \begin{verbatim} 1161 INPUT: 1162 var1 -- <type> (default: ...) description 1163 var2 -- <type> (default: ...) description 1164 etc. 1165 1166 OUTPUT: 1167 <type> -- description of return value 1 1168 <type> -- description of return value 2 1169 etc. 1170 \end{verbatim} 1171 1172 This is a placeholder of a "Perfect example of the above".\todo{ 1173 I=ifti propose everywhere in this document when there is a template, it is followed by a path/filename where in one can find a "perfect" example.} 1174 1175 For consistency you should use the above format, which is used 1176 throughout the system. Do not just describe the input and output 1177 in long sentences. For methods of a class, whether or not 1178 to describe self is up to you; this can be useful if self must 1179 have some special properties. Also, you do not have to 1180 specify the types of the input and output arguments 1181 precisely; e.g., use ``integer'' for an int or long or 1182 \sage \code{Integer}. 1183 1184 Every function should have at least one, and preferably many, examples 1185 that illustrate its usage. These examples must exactly match the 1186 output of an actual running copy of Python. To test whether this is 1187 the case, run the program \code{sage -t} with your Python or Cython 1188 file as unique argument (see Chapter~\ref{ch:testing} for more details). 1189 1190 For more details, see Section~\ref{sec:conv-docs}.\todo{redundancy \S 2.4 and \S4.1} 1191 1192 \section{Extra Documentation} 1193 You might also write additional documentation in \Latex, which is not to 1194 be part of any source code file. The examples in this documentation 1195 should be contained in verbatim environments. The examples should be 1196 tested using \code{sage-testtex}, which you run on the \Latex file. Put 1197 \%skip on the line preceding the environment to skip testing of an 1198 example. 1199 1200 Note that \code{sage-testtex} is completely different than \code{sage-doctest}. It extracts the documentation and feeds it to a 1201 running instance of \SAGE. 1202 1203 1204 \section{Automated Testing}\label{ch:testing} 1205 This section describes automated testing of test files of the 1206 following types: \code{.py .pyx .sage .tex}. Use \code{sage -t 1207 <file>} to test that the examples in \code{<file>} behave exactly as 1208 claimed. See Section~\ref{sec:conv-docs} for a discussion of how to 1209 include examples in documentation strings and what conventions to 1210 follow --- Section ~\ref{sec:Further-conventions}. 1211 1212 \subsection{Testing .py, .pyx and .sage Files}\label{sec:test:python} 1213 Run \code{sage -t <filename.py>} to test that all code examples in 1214 \code{filename.py}. Similar remarks apply to \code{.sage} and 1215 \code{.pyx} files. 1216 1217 \begin{verbatim} 1218 sage -t [--verbose] [--optional] [files and directories ... ] 1219 \end{verbatim} 1220 1221 When you run \code{sage -t <filename.py>}, \sage makes a copy of 1222 \code{<filename.py>} with all the \code{sage} prompts replaced by 1223 \code{{>}{>}>}, then uses the standard Python doctest framework to test 1224 the documentation. More precisely, 1225 the Python script \code{local/bin/sage-doctest} implements 1226 documentation testing. It does the following when asked 1227 to test a file \code{foo.py} or \code{foo.sage}. 1228 \begin{enumerate} 1229 1230 \item Creates the directory \code{.doctest} if it doesn't 1231 exist and the file \code{.doctest/foo.py}. 1232 1233 \item The file \code{.doctest_foo.py} contains functions for 1234 each docstring in \code{foo.py}, but with all \sage preparsing 1235 applied and with \code{from sage.all import *} at the top. 1236 The function documentation is thus standard Python with 1237 \code{{>}{>}>} prompts. 1238 1239 \item \code{sage-doctest} then runs \sage's Python interpreter 1240 on \code{.doctest_foo.py}. 1241 \end{enumerate} 1242 1243 \note{{\em Note that line numbers in the errors 1244 you might see apply to the file \code{.doctest_foo.py}, not 1245 to the original file \code{foo.py}!} If you get errors, 1246 feel free to inspect \code{.doctest_foo.py}; it is never 1247 automatically deleted by \sage.} 1248 1249 Your file passes these tests if the code in it will run when entered 1250 at the \code{sage:} prompt with no special imports. Thus users are 1251 guaranteed to be able to exactly copy code out of the examples you 1252 write for the documentation and have them work. 1253 1254 %skip 1255 \begin{verbatim} 1256 1257 William: 1258 1259 I changed the doctest behavior recently so that now you can doctest 1260 files anywhere (not necessary in \SAGE), and type e.g., 1261 1262 sage: from foo import * 1263 sage: ... 1264 1265 in the doctest blocks and it will work, even if foo.py isn't part of the 1266 \SAGE library. It's much better now. Feel free to change the reference 1267 manual to reflect this. 1268 1269 \end{verbatim} 1270 1271 1272 \subsection{Testing \Latex Documentation} 1273 1274 Run \code{sage -t <filename.tex>} to test the examples in 1275 verbatim environments in \Latex documentation. Put 1276 \code{\%skip} 1277 on the line right before a verbatim environment to have that 1278 example skipped when testing the file. 1279 1280 \sage creates a file \code{.doctest_filename.py} and tests 1281 it just as for \code{.py}, \code{.pyx} and \code{.sage} 1282 files. In order to skip testing of a block of code 1283 in a verbatim environment, put the \Latex comment 1284 \code{\%skip} on the previous line. 1285 1286 One difference is that in \Latex files one 1287 often inserts explanatory texts between different 1288 verbatim environments. To link together verbatim 1289 environments use the \code{\%link} comment. For 1290 example\todo{I have changed verbatim to verbatim0 here 1291 since I don't know how to include end{verbatim} in a 1292 verbatim environment.} 1293 %skip 1294 \begin{verbatim} 1295 \begin{verbatim} 1296 sage: a = 1 1297 \end{verbatim_}%link 1298 1299 Next we add 1 to \code{a}. 1300 %link 1301 \begin{verbatim} 1302 sage: 1 + a 1303 2 1304 \end{verbatim_} 1305 \end{verbatim} 1306 1307 See \code{SAGE_ROOT/doc/tut/tut.tex} for many examples 1308 of how to include automated testing in \Latex documentation 1309 for \sage. 1310 1311 1312 1313 \section{Modifying the \SAGE Manuals} 1314 We use the tutorial as an example. 1315 1316 You just modify the file \code{SAGE_ROOT/devel/doc/tut/tut.tex} To 1317 build a test copy of the tutorial with your changes, type 1318 \code{./build_dvi} in the \code{devel/doc/tut} directory. You'll get 1319 a file \code{tut.dvi}. 1320 You can also build HTML versions of everything by typing \code{make 1321 html} in the \code{devel/doc} directory. Build pdfs with \code{make 1322 pdf}. \todo{You could also do \code{./build_pdf} to create a pdf directly from the tex file.} 1323 1324 It is a good idea to type \code{sage -t tut.tex} to make sure any 1325 examples you added to the tutorial work as claimed. 1326 1327 The \code{SAGE_ROOT/devel/doc} directory is a Mercurial repository 1328 (see Chapter~\ref{ch:mercurial}), so you can see exactly what changes 1329 you've made to the documentation, make patches of what you've done 1330 available, etc. You can do this from within \sage by typing 1331 \code{hg_doc.[tab]}. 1332 1333 Imagine you have edited this Programming Manual found in \code{devel/doc/prog}. 1334 Before you started you did a pull to be sure your docs are up to date: 1335 \todo{Give some examples of usage} 1336 1337 %skip 1338 \begin{verbatim} 1339 sage: hg_doc.pull() 1340 cd "/home/jaap/sage/devel/doc" && hg status 1341 cd "/home/jaap/sage/devel/doc" && hg status 1342 cd "/home/jaap/sage/devel/doc" && hg pull -u http://sagemath.org/sage//hg//doc-main 1343 pulling from http://sagemath.org/sage//hg//doc-main 1344 searching for changes 1345 adding changesets 1346 adding manifests 1347 adding file changes 1348 added 3 changesets with 3 changes to 1 files 1349 1 files updated, 0 files merged, 0 files removed, 0 files unresolved 1350 If it says use 'hg merge' above, then you should 1351 type hg_sage.merge(), where hg_sage is the name 1352 of the repository you are using. This might not 1353 work with the notebook yet. 1354 \end{verbatim} 1355 1356 After you made many changes, you want to commit them: 1357 %skip 1358 \begin{verbatim} 1359 sage: hg_doc.commit() 1360 cd "/home/jaap/downloads/sage-1.6/devel/doc" && hg diff | less 1361 cd "/home/jaap/downloads/sage-1.6/devel/doc" && hg commit 1362 \end{verbatim} 1363 1364 You saw the differences piped through \code{diff}, you were asked 1365 to comment: type 'i' or 'a', your comment, hit 'Escape' and type ':wq". 1366 1367 Now you can make a patch 'patchname'. e.g.: 1368 %skip 1369 \begin{verbatim} 1370 sage: hg_doc.send('prog20070118.hg') 1371 Writing to /home/jaap/sage/prog20070118.hg 1372 cd "/home/jaap/sage/devel/doc" && hg bundle tmphg http://sagemath.org/sage//hg//doc-main 1373 searching for changes 1374 Successfully created hg patch bundle /home/jaap/sage/prog20070118.hg 1375 \end{verbatim} 1376 1377 Send the file 'patchname' to the \code{sage-devel} Google group. 1378 1379 1380 \chapter{Writing Optimized Compiled Code With \SAGE} 1381 This chapter is about how to write fast compiled optimized code for 1382 \sage. It was written mainly by William Stein with contributions 1383 from David Harvey, Martin Albrecht, David Joyner, Alex Clemesha. 1384 Some of the text is closely copied or translated (to mathematical 1385 language) from Greg Ewing's original Pyrex manual. 1386 1387 This chapter assumes that you have a C compiler suite (e.g., GCC) 1388 installed on your computer, since you should not write compiled 1389 extension code on a computer without a C compiler. If you are using 1390 OS X, download and install the latest version of XCode.\footnote{The 1391 download is about 1GB, and you must sign up for the Apple Developer 1392 Network. \url{http://developer.apple.com/tools/xcode/}} 1393 For Linux, use 1394 your package manager to install GCC. If you're using Microsoft 1395 Windows, then you should be using \sage via colinux, which includes 1396 GCC by default. 1397 1398 1399 %[[General discussion of when to use compiled code versus interpreted 1400 %code.]] 1401 1402 Interpreted code: way easier to debug, 1403 trace, document, structure (multiple inheritence), etc.; 1404 good when few for loops; lots of structure and calls 1405 to compiled code. 1406 1407 Compiled code: two reasons to use---make fast code and 1408 make optimal use of libraries written in a compiled language; 1409 hard to write and debug; takes longer to 1410 write. The methods explained in this chapter can produce 1411 code where speed is only limited by your choice of algorithm. 1412 1413 1414 1415 {\bf TODO:} 1416 \begin{enumerate} 1417 \item Module name and file name must agree! 1418 %[(Check this at build time?)] 1419 1420 \end{enumerate} 1421 1422 \section{Introduction} 1423 This chapter describes how to write \SAGE programs using Cython, which 1424 is a language for writing Python extension modules that is a very 1425 slightly variant of Pyrex (we will often refer to it as ``Pyrex'' 1426 still). 1427 1428 The primary goal of this chapter is to provide excellent documentation 1429 about how to use Cython to write very fast code in the context of 1430 \SAGE. (Currently it is a mess.) This chapter is self contained in 1431 that the reader should not have to look at any of the documentation 1432 distributed with Pyrex. The reader is urged to at some point read the 1433 Python/C API Reference manual. 1434 1435 There is also much to be learned from the Pyrex 1436 mailing list: 1437 1438 \url{http://lists.copyleft.no/mailman/listinfo/pyrex}. 1439 1440 {\bf Acknowledgement:} The existence of Greg Ewing's Pyrex is 1441 one of the primary reasons Stein chose Python as the implementation 1442 language for \SAGE. It is a brilliantly useful tool. 1443 1444 %\cite{pyrex:list}. 1445 1446 %A list of quality open source projects that 1447 %use Pyrex: soya3d, ... [[add more here]] 1448 1449 1450 \subsection{Interpreted and Compiled Code} 1451 Every serious computer algebra system, e.g., MAGMA, PARI, Mathematica, 1452 Maple, GAP, Singular, etc., is implemented as a combination of 1453 compiled and interpreted code. For Mathematica, Singular, MAGMA, and 1454 PARI, most of the implementation is in compiled C (or C++) code; some 1455 of these systems tend to be very optimized (subject to the constraints 1456 of the algorithms they implement). In contrast, Maple and GAP have a 1457 relatively small compiled ``kernel'' that defines the underlying 1458 programming language; most of the system is then implemented in this 1459 language. If you do benchmarks you'll discover that Mathematica is 1460 {\em much} faster than Maple at some basic operations. Likewise, 1461 benchmarks reveal that MAGMA is often faster than GAP. 1462 %[[give 1463 %examples that illustrate each of these points. E.g., basic 1464 %arithmetic where maple is pitiful, computation of Bernoulli 1465 %numbers where PARI crushes all others.]] 1466 1467 This fusion of interpreted and compiled code is extremely natural for 1468 mathematics software; some algorithms are much better implemented in 1469 an interpreter because all time-critical steps involve low level 1470 arithmetic --- other algorithms, e.g., matrix multiplication, must be 1471 implemented in a compiled language in order to perform optimally. 1472 Also, existing compiled libraries are sometimes of very high quality, 1473 and a compiled language is needed to create the best possible 1474 interface to such libraries. It's crucial that both approaches to 1475 programming be fully supported. When deciding how to implement \SAGE, 1476 I searched for months for an environment that could support both 1477 approaches. Python and Cython provide exactly this in a way that I 1478 believe is much easier conceptually than the implementation models of 1479 any of the other systems mentioned above. 1480 1481 \subsection{Why Cython is the Best Available Option for \SAGE} 1482 The software listed in this section is used either 1483 to speed up Python programs or to make code written in 1484 C++, C, FORTRAN, etc., available in Python. This section briefly 1485 explains what each tool does and why it is {\em not} included 1486 in \SAGE or used in the implementation of the \SAGE library. 1487 1488 1489 \subsubsection{SWIG} 1490 SWIG (Standard Wrapper and Interface Generator) 1491 takes C++ (and C, etc.) libraries, and generates 1492 wrapper code so that the code can be used from 1493 within Python (and in several other interpreters). One 1494 writes interface files that describe how to make Python 1495 objects usable from your C++ code, and conversely what 1496 transformations C++ objects should undergo to be usable from Python. 1497 1498 In 2004, Stein considered using SWIG for \SAGE extensively, and implemented 1499 several basic types using this model. The idea was to 1500 write code in C++ for \SAGE that needed to be fast, then wrap it in 1501 SWIG. This ground to a halt, because the result was not 1502 {\em sufficiently} fast. First, there is overhead when writing code 1503 in C++ in the first place. Second, SWIG generates several layers of 1504 code between Python and the code that does the actual work; for 1505 implementing any of \SAGE's numerous basic arithmetic types, 1506 e.g., integers, the overhead is completely unacceptable, at 1507 least if one wants code that is as fast as Magma and PARI. For 1508 example, in my initial implementation of integer arithmetic using 1509 SWIG and GMP, there were the following {\em six} levels between the 1510 \sage interpreter and the GMP C library: 1511 \begin{center} 1512 \begin{tabular}{|l|}\hline 1513 Python code to provide a clean interface\\\hline 1514 SWIG Autogenerated Python code \\\hline 1515 SWIG Autogenerated C++ extension code \\\hline 1516 Handcode C++ Integer class\\\hline 1517 GMP's C++ Interface\\\hline 1518 GMP C Library\\\hline 1519 \end{tabular} 1520 \end{center} 1521 In this model, just multiplying two integers is extremely complicated, 1522 and slow. Moreover, writing the SWIG interface files was 1523 painful, since general Python types are difficult to work with from 1524 C++ code. In contrast, with Cython the situation is as follows 1525 (see {\tt integer.pyx}): 1526 \begin{center} 1527 \begin{tabular}{|l|}\hline 1528 Cython generated C code that provides everything needed\\\hline 1529 GMP C Library\\\hline 1530 \end{tabular} 1531 \end{center} 1532 Another important problem with using SWIG is that 1533 developers have to know C++ well and have the mental dexterity 1534 to easily switch back and forth between C++ and Python. 1535 Stein does not have this ability. 1536 1537 \subsubsection{Boost} 1538 Like SWIG, Boost takes C++ code and generates C/C++ extension code and Python 1539 wrappers around that extension code, which one can then use from the 1540 Python interpreter. \SAGE does not use Boost because it is large and 1541 difficult to redistribute and it is meant much more for wrapping 1542 existing C++ libraries. Boost also uses a very complicated and 1543 specialized build system. 1544 1545 \subsubsection{Psycho} 1546 Psycho is a Python library that tries to guess the types of variables 1547 in a running Python program, then generate compiled extension code 1548 {\em on the fly} to optimize the running program. \SAGE does not use 1549 Psycho, because it only runs on Intel x86 machines, 1550 %[[what about 1551 %OS X macs, windows?]] 1552 it has large and 1553 unpredictable memory requirements, and so much behind-the-scenes 1554 guessing at this level for a serious research mathematics tool is 1555 inappropriate. 1556 1557 %[[Nonetheless: quick tutorial and examples]] 1558 1559 Moreover, in my own experience with \sage, the 1560 speedups from Psycho were not significant, except on code that had an 1561 obvious C equivalent. Such code can be easily rewritten in Cython, 1562 which always results in something that is faster, more robust, and 1563 more reliable than what Pyscho produces, and which can be debugged 1564 using gdb. 1565 1566 Though Psycho is not included with or used by \SAGE, 1567 it is possible to try out Psycho by 1568 installing it and putting the appropriate line at the top of a file. 1569 1570 %[[put a mathy example in with timings]] 1571 1572 \subsubsection{Ctypes} 1573 Ctypes is included standard in Python 2.5, so it is clearly useful to 1574 many Python users. It allows one to call library 1575 functions defined in shared object libraries directory from 1576 interpreted Python code. For the core of \SAGE 1577 we have not found a use for this yet. 1578 First, when implementing a basic type that needs to 1579 be very fast, e.g., GMP integers, the entire implementation absolutely 1580 must be compiled -- writing it as a combination of Python and C 1581 library calls will be way too slow. Second, it is often very painful 1582 to convert Python data structures to C data structures and conversely, 1583 and there are many opportunities for memory leaks and core dumps. 1584 1585 %[[example. like the ``install'' command in PARI.]] 1586 1587 \section{Examples} 1189 Python is an interpreted language and has no declared data types for 1190 variables. These features make it easy to write and debug, but Python 1191 code can sometimes be slow. Cython code can look a lot like Python, 1192 but it gets translated into C code (often very efficient C code) and 1193 then compiled. Thus it offers a language which is familiar to Python 1194 developers, but with the potential for much greater speed. 1588 1195 1589 1196 There are several ways to create and build Cython code in \SAGE. 1590 1197 \begin{enumerate} … … 1600 1207 for you to use in the notebook. Also, the output cell has 1601 1208 a link to the C program that was compiled to create 1602 1209 the .so file. 1603 \item Below we will denote notebook sessions as follows:1604 \begin{verbatim}1605 {{{1606 INPUT TEXT1607 ///1608 OUTPUT TEXT1609 }}}1610 ...1611 \end{verbatim}1612 The triple braces delimit cells. You should {\em not}1613 actually type in the curly braces or triple forward1614 slashes.1615 1616 1210 \end{enumerate} 1617 1211 1618 1212 \item Create an .spyx file and attach or load it 1619 1213 from the command line. 1620 1214 This is similar to creating a {\tt \%cython} cell 1621 1215 in the notebook but works completely from the command 1622 line .1216 line (and not from the notebook). 1623 1217 1624 \item Create a .pyx file in the \SAGE library, add it to {\tt 1625 SAGE_ROOT/devel/sage/setup.py} (note -- you have to follow the 1626 template of how something else in there is done -- we simply haven't 1627 had the time to properly document this process yet!), and type {\tt 1628 sage -br} to build it. This is how much of the core arithmetic in 1629 the \SAGE library is implemented. 1630 1631 1218 \item Create a .pyx file in the \SAGE library and add a listing for it 1219 to the variable \code{ext_modules} in the file 1220 \code{SAGE_ROOT/devel/sage/setup.py}. For example, the file 1221 \code{SAGE_ROOT/devel/sage/sage/graphs/chrompoly.pyx} has lines 1222 %skip 1223 \begin{verbatim} 1224 Extension('sage.graphs.chrompoly', 1225 ['sage/graphs/chrompoly.pyx'] 1226 ), \ 1227 \end{verbatim} 1228 in \code{setup.py}. Then type \code{sage -ba} to build \sage with the 1229 new code. 1632 1230 \end{enumerate} 1633 1231 1634 1232 \subsection{Attaching or Loading {\tt .spyx} Files} 1233 1635 1234 The easiest way to try out Cython without having to 1636 1235 learn anything about distutils, etc., is to create 1637 1236 a file with the extension {\tt spyx}, which stands 1638 for ``SAGE Pyrex'' . Try it now!1237 for ``SAGE Pyrex'': 1639 1238 1640 \subsection{Using the Command Line and a File}1641 1239 \begin{enumerate} 1642 \item Create a file {\tt hello.spyx}.1240 \item Create a file \code{power2.spyx}. 1643 1241 \item Put the following in it: 1644 1242 \begin{verbatim} 1645 def hello(name): 1646 """ 1647 Print hello with the given name. 1648 """ 1649 print("Hello %s"%name) 1243 def is2pow(n): 1244 while n != 0 and n%2 == 0: 1245 n = n >> 1 1246 return n == 1 1650 1247 \end{verbatim} 1651 1248 \item Start the \SAGE command-line interpreter and load 1652 1249 the spyx file (this will fail if you do not have a C compiler 1653 1250 installed). 1654 1251 %skip 1655 1252 \begin{verbatim} 1656 sage: load " hello.spyx"1657 Compiling hello.spyx...1658 sage: hello('World')1659 Hello World 1253 sage: load "power2.spyx" 1254 Compiling power2.spyx... 1255 sage: is2pow(12) 1256 False 1660 1257 \end{verbatim} 1661 1258 \end{enumerate} 1662 Note that you can change {\tt hello.spyx}, then load1259 Note that you can change \code{power2.spyx}, then load 1663 1260 it again and it will be recompiled on the fly. 1664 You can also attach {\tt hello.spyx} so it is reloaded1665 whenever you make changes .1261 You can also attach \code{power2.spyx} so it is reloaded 1262 whenever you make changes: 1666 1263 %skip 1667 1264 \begin{verbatim} 1668 sage: attach " hello.spyx"1265 sage: attach "power2.spyx" 1669 1266 \end{verbatim} 1670 1671 \subsection{An Optimization Example} 1672 Imagine you are writing a program that frequently computes {\tt a*b + c} 1673 for $a,b,c$ \SAGE integers. For example, you want to 1674 do the following computation (timings on a Macbook Pro 2Ghz 1675 under OS X): 1267 Cython is used for its speed. Here is a timed test on a 2.4 GHz iMac: 1676 1268 %skip 1677 1269 \begin{verbatim} 1678 sage: a = 939393; b = 19393; c = 393902831679 sage: time for _ in range(10000): a = a*b + c 1680 CPU time s: user 0.33 s, sys: 0.21 s, total: 0.53s1270 sage: time [n for n in range(10^5) if is2pow(n)] 1271 [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536] 1272 CPU time: 0.06 s, Wall time: 0.06 s 1681 1273 \end{verbatim} 1682 1683 There is no ``multiply then add'' operation built into \SAGE, and 1684 you're very frustrated because you really want this to be as fast as 1685 possible. There is possibly significant overhead in the computation 1686 of {\tt a*b + c} in Python. 1687 1688 First create a new compiled ``multiply and add'' command in \SAGE. 1689 1690 %[[todo: change example below to use {\tt\_\_new\_\_}.]] 1691 \begin{verbatim} 1692 cimport sage.rings.integer 1693 def muladd(sage.rings.integer.Integer a, 1694 sage.rings.integer.Integer b, 1695 sage.rings.integer.Integer c): 1696 """ 1697 Compute a*b + c 1698 """ 1699 cdef sage.rings.integer.Integer d 1700 d = sage.rings.integer.Integer() 1701 mpz_mul(d.value, a.value, b.value) 1702 mpz_add(d.value, d.value, c.value) 1703 return d 1704 \end{verbatim} 1705 1706 Do this either by putting the above code in a spyx 1707 file or pasting it into the notebook in a cell that 1708 starts with {\tt\%cython}. 1709 1710 Now we can do the above computation more quickly: 1274 Now, the code in the file \code{power2.spyx} is valid Python, and if 1275 we copy this to a file \code{powerslow.py} and load that, we get the 1276 following: 1711 1277 %skip 1712 1278 \begin{verbatim} 1713 sage: a = 939393; b = 19393; c = 39390283 1714 sage: time for _ in range(10000): a = muladd(a,b,c) 1715 CPU times: user 0.11 s, sys: 0.05 s, total: 0.15 s 1279 sage: load "powerslow.py" 1280 sage: time [n for n in range(10^5) if is2pow(n)] 1281 [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536] 1282 CPU time: 1.80 s, Wall time: 1.84 s 1716 1283 \end{verbatim} 1284 By the way, we could gain even a little more speed -- cutting it down 1285 to 0.03 s -- with the Cython version with a type declaration, by 1286 changing \code{def is2pow(n):} to \code{def is2pow(unsigned int n):}. 1717 1287 1718 To squeeze out even more performance we can put the1719 entire loop in a single function.1720 \begin{verbatim}1721 cimport sage.rings.integer1722 def muladdloop(sage.rings.integer.Integer a,1723 sage.rings.integer.Integer b,1724 sage.rings.integer.Integer c,1725 int n):1726 cdef int i1727 cdef mpz_t t1728 cdef sage.rings.integer.Integer aa1729 aa = sage.rings.integer.Integer(a)1730 mpz_init(t)1731 1288 1732 for i from 0 <= i < n:1733 mpz_mul(t, aa.value, b.value)1734 mpz_add(aa.value, t, c.value)1735 1289 1736 mpz_clear(t) 1737 return aa 1738 \end{verbatim} 1290 \section{Other Languages} 1739 1291 1740 We then have 1741 %skip 1742 \begin{verbatim} 1743 sage: a = 939393; b = 19393; c = 39390283 1744 sage: time z=muladdloop(a,b,c,10000) 1745 CPU times: user 0.11 s, sys: 0.00 s, total: 0.11 s 1746 \end{verbatim} 1747 This is a little faster. %[[no system overhead -- what's going on?]] 1292 Since \sage is based on Python, it interfaces with C and C++, as well 1293 as other languages. See the Python documentation at 1294 \url{http://www.python.org/doc/} for more details; in particular, 1295 ``Extending and Embedding the Python Interpreter'', available at 1296 \url{http://docs.python.org/ext/ext.html}, describes how to write C or 1297 C++ modules for use in Python. 1748 1298 1749 \subsection{A simple loop example: Sum of squares}1750 1751 The following \SAGE worksheet illustrates the above1752 sums of squares examples in the \SAGE Notebook.1753 1754 Note, we use the notation:1755 1756 %skip1757 \begin{verbatim}1758 {{{1759 INPUT TEXT1760 ///1761 OUTPUT TEXT1762 }}}1763 \end{verbatim}1764 to denote cells in the notebook.1765 1766 %[[todo put a check to avoid overflow below. maybe switch1767 %to using long long, or also use that...]]1768 \begin{verbatim}1769 {{{1770 two = int(2)1771 def sumsquarespy(n):1772 return sum(i**two for i in xrange(1,n+1))1773 }}}1774 {{{1775 time v=[sumsquarespy(100) for _ in xrange(10000)]1776 ///1777 #> CPU time: 0.49 s, Wall time: 0.49 s1778 }}}1779 {{{1780 %cython1781 # Next in cython.1782 def sumsquares(int n):1783 cdef int i, j1784 j = 01785 for i from 1 <= i <= n:1786 j = j + i*i1787 return j1788 }}}1789 {{{1790 time v=[sumsquares(100) for _ in xrange(10000)]1791 ///1792 CPU time: 0.06 s, Wall time: 0.06 s1793 }}}1794 \end{verbatim}1795 1796 \subsection{Example: 2 Power}1797 The following \SAGE notebook session illustrates how to implement1798 in Cython (entirely using the notebook) a function to quickly tell1799 if a C integer is a power of $2$. The resulting function is over 2001800 times faster in Cython than in Python, though the actual code1801 is {\em identical}.1802 \begin{verbatim}1803 {{{1804 %cython1805 def is2pow(unsigned int n):1806 while n != 0 and n%2 == 0:1807 n = n >> 11808 return n == 11809 }}}1810 \end{verbatim}1811 \begin{verbatim}1812 {{{1813 time [n for n in range(10^5) if is2pow(n)]1814 ///1815 [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536]1816 CPU time: 0.05 s, Wall time: 0.06 s1817 }}}1818 \end{verbatim}1819 \begin{verbatim}1820 {{{1821 # The same program but in Python:1822 def is2pow(n):1823 while n != 0 and n%2 == 0:1824 n = n >> 11825 return n == 11826 }}}1827 \end{verbatim}1828 \begin{verbatim}1829 {{{1830 time [n for n in range(10^5) if is2pow(n)]1831 ///1832 [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536]1833 CPU time: 13.04 s, Wall time: 15.08 s1834 }}}1835 \end{verbatim}1836 \begin{verbatim}1837 {{{1838 13.04/0.051839 ///1840 260.799999999999951841 }}}1842 \end{verbatim}1843 1844 %\subsection{Adding to \SAGE}1845 1846 %\subsection{Your Own {\tt setup.py}}1847 1848 \section{Cython Manual}1849 1850 \subsection{Memory Usage}1851 Here is the memory usage of an integer mod int object on a 32-bit system:1852 \begin{verbatim}1853 4 an int (the underlying value)1854 4 pointer to the parent1855 4 pointer to the modulus1856 4 pointer to the function call table1857 8 PyObject_HEAD (pointer to type object and reference count)1858 \end{verbatim}1859 So a single integer mod takes 24 bytes.1860 1861 {\em IDEA: For a small field, one could store every element, then just have1862 lots of copies of pointers to the same thing, which would save a ton1863 of memory. }1864 1865 \subsection{Forward-declaring extension types}1866 Extension types can be forward declared, which is necessary, e.g.,1867 if you have two extension types that need to refer to each other.1868 For example,1869 \begin{verbatim}1870 cdef class MyRing # forward declaration1871 1872 cdef class MyElement:1873 cdef MyRing parent1874 1875 cdef class MyRing:1876 cdef MyElement basis1877 \end{verbatim}1878 1879 If you forward declare an extension type that has a base class, you1880 must specify the base class in both the forward declaration and its1881 subsequent definition. For example,1882 1883 \begin{verbatim}1884 cdef class A(B)1885 1886 ...1887 1888 cdef class A(B):1889 # attributes and methods1890 \end{verbatim}1891 1892 \subsection{Using cdef Module-Level Functions in Other Modules}1893 Suppose there is a cdef function in a Cython file \code{A.pyx} that is1894 not a method of a class, but you would like to use it from another1895 Cython module. There are (at least) two options.1896 One option is to make the functions methods of a cdef'd1897 class, then instantiate an object of that type in the other file1898 (this has no weird linking issues).1899 Another is to use ``public extern'' and link in the whole module, which we describe1900 below.1901 1902 For example, the file \code{A.pyx} might contain1903 \begin{verbatim}1904 cdef int sqr_int(int x):1905 return x*x1906 \end{verbatim}1907 You would like to call this C-level method directly from a different Cython module.1908 To do so, {\em it is crucial} to declare the method public instead:1909 \begin{verbatim}1910 cdef public int sqr_int(int x):1911 return x*x1912 \end{verbatim}1913 Then create a file \code{A.pxd} that contains1914 \begin{verbatim}1915 cdef extern int sqr_int(int x)1916 \end{verbatim}1917 Finally, to use \code{sqr_int} in another Cython module \code{B.pyx},1918 do the following:1919 \begin{verbatim}1920 cimport sage.rings.A # this is just an example absolute path1921 def myfunction():1922 print sage.rings.A.sqr_int(5)1923 \end{verbatim}1924 Any time you access one Cython module from another one,1925 it's necessary that the compiler know to link in the other Cython1926 module (\code{A.pyx} in our example). Tell it to link1927 in \code{A.pyx} by including \code{A.pyx}1928 in the corresponding entry in \code{setup.py}. That1929 entry should look something like this:1930 \begin{verbatim}1931 Extension('sage.modular.B',1932 ['sage/modular/B.pyx', 'sage/rings/A.pyx'])1933 \end{verbatim}1934 \note{Linking as above means an entire copy of all the code in \code{A.pyx}1935 is compiled into the module for \code{B.pyx}, so \code{B.so} will be larger,1936 and the C-level function you're accessing will {\em not} have access1937 to any module-scope variables in \code{A.pyx}. This can lead to weird1938 core dumps, depending on the nature of your function.}1939 1940 Cython will generate a file \code{A.pxi} that also declares foo. You1941 can ignore that file.1942 1943 {\bf NOTE:} Alternatively, you could put \code{include "path/to/A.pxi"}1944 in \code{B.pyx} and and directly use \code{sqr_int} in \code{B.pyx} with1945 no scoping:1946 \begin{verbatim}1947 def myfunction():1948 print sqr_int(5)1949 \end{verbatim}1950 This is not recommended because it is confusing and1951 unpythonic to not have the scoping information.1952 1953 1954 \subsection{Making it possible to create weak references to extension classes}1955 By default, extension types do not support having weak references made1956 to them. You can enable weak referencing by declaring a C attribute of1957 type object called \code{__weakref__}. For example,1958 \begin{verbatim}1959 cdef class MyRing:1960 """1961 There is supposed to be at most one instance of this ring, so1962 we keep a weak reference pointer to it, so it will be deleted1963 from memory when there are no more user references to it.1964 """1965 cdef object __weakref__1966 \end{verbatim}1967 1968 \subsection{Public and Private Variables}1969 To expose a variable to external Python, use \code{cdef public}, e.g.,1970 \begin{verbatim}1971 cdef class MyRing:1972 cdef public Integer characteristic1973 \end{verbatim}1974 Use readonly if you want to enable only readonly access.1975 \begin{verbatim}1976 cdef class MyRing:1977 cdef readonly Integer characteristic1978 \end{verbatim}1979 1980 \subsection{Arbitrary Attributes}1981 Reminder: Like builtin types, Cython extension types don't have a1982 \code{__dict__} field, so you cannot assign arbitrary attributes to1983 them. That said, it is just a few lines to implement attribute1984 assignment, as illustrated by the following notebook session:1985 \begin{verbatim}1986 {{{1987 %cython1988 cdef class X:1989 cdef object __dict__1990 def __init__(self):1991 self.__dict__ = {}1992 def __setattr__(self, attr, val):1993 self.__dict__[attr] = val1994 def __getattr__(self, attr):1995 try:1996 return self.__dict__[attr]1997 except KeyError:1998 raise AttributeError, "object has no attribute '%s'"%attr1999 ///2000 }}}2001 {{{2002 x = X()2003 ///2004 }}}2005 {{{2006 x.a = 52007 ///2008 }}}2009 {{{2010 x.a2011 ///2012 52013 }}}2014 {{{2015 x.b2016 ///2017 Traceback (most recent call last): x.b2018 File "/Volumes/HOME/sage-stable/local/lib/python2.5/", line 1, in <module>2019 2020 File "/Volumes/HOME/.sage//spyx/sage20/sage20_0.pyx", line 14, in sage20_0.X.__getattr__2021 raise AttributeError, "object has no attribute '%s'"%attr2022 AttributeError: object has no attribute 'b'2023 }}}2024 \end{verbatim}2025 2026 \subsection{Special Methods}2027 Certain special methods {\em must} be defined in every extension class2028 in order to work. In particular, they do not derive from higher classes.2029 This can be very confusing if you are not aware of it. For2030 example, suppose you define two classes $A$ and $B$ as follows:2031 \begin{verbatim}2032 cdef class A:2033 def __hash__(self):2034 return 02035 2036 cdef class B(A):2037 def __repr__(self):2038 return "I am class B"2039 \end{verbatim}2040 Then $B$ will {\em not} inherit the \code{__hash__} method2041 from class $A$. If you want $B$ to be hashable, you {\em must}2042 define a hash method explicitly for $B$, e.g.,2043 \begin{verbatim}2044 cdef class B(A):2045 def __hash__(self):2046 return A.__hash__(B)2047 def __repr__(self):2048 return "I am class B"2049 \end{verbatim}2050 2051 I do not know exactly which special methods do not get inherited.2052 Definitely \code{__hash__} and \code{__richcmp__} do not.2053 2054 2055 2056 %\section{Gotchas}2057 2058 %\subsection{How C extension classes different from Python classes}2059 2060 %\subsection{When to use {\tt cdef class} versuses {\tt def class}2061 %in Cython}2062 %def allows multiple inheritance, but no c-level data structures.2063 %Maybe creation is faster. What exactly is the deal here?!2064 2065 %\subsection{\SAGE's Pyrex is Patched}2066 2067 %[[Record exactly how and why here. Post this to the2068 %mailing list every once in a while. ]]2069 2070 2071 %\subsection{There is no +=}2072 %[[solution make a C macro. need a sage.pxi that has all special2073 %convenience code like this in it, and is automatically included.]]2074 2075 %\subsection{Notation for C for Loops}2076 2077 %\subsection{Typos in Function Names and Variables}2078 2079 \subsection{Doctests}2080 If \code{x.pyx} is a Cython file, then you can do2081 \begin{verbatim}2082 sage -t x.pyx2083 \end{verbatim}2084 to test all the examples in the documentation strings2085 in \code{x.pyx}.2086 2087 \subsection{Signal Handling}2088 Suppose you have a Cython function like2089 \begin{verbatim}2090 def foo():2091 cdef int n2092 # Some long running code that gets compiled to pure C, e.g.,2093 while True:2094 n = n + 12095 2096 \end{verbatim}2097 If you call foo from the \SAGE interpreter, then when \SAGE2098 gets to the ``chunk of long-running C code'' pressing control-C2099 will not interrupt the calculation. The only way to interrupt2100 the calculation is to completely kill your running copy of2101 \SAGE. What a wasteful pain---surely there is a better way!2102 2103 Indeed, if you simply rewrite the above code as follows, then2104 suddenly control-C will work fine!2105 \begin{verbatim}2106 def foo():2107 cdef int n2108 _sig_on2109 while True:2110 n = n + 12111 _sig_off2112 \end{verbatim}2113 \note{In a .pyx file which will be part of the \SAGE library,2114 you have to explicitly put something like2115 \code{include "../ext/interrupt.pxi"} in your file,2116 where you give in quotes the relative directory2117 to \code{SAGE_ROOT/devel/sage/sage/ext/interrupt.pxi}.}2118 2119 In addition to SIGINT, any code wrapped in \code{_sig_on} and2120 \code{_sig_off} also traps SIGABRT, SIGALRM, SIGSEV, and SIGFPE.2121 2122 {\bf IMPORTANT NOTES:}2123 \begin{enumerate}2124 \item These {\em must} always come in pairs. E.g., if you have just2125 a \code{_sig_off} without a corresponding \code{_sig_on}, then control-C2126 later in the interpreter will segfault!2127 2128 \item Do {\em not} put these in the \code{__init__} method of2129 a Cython extension class, or you'll get crashes.2130 2131 \item Do not put these around any Cython code that calls into2132 the Python API (i.e., the generated C code should have no2133 calls to the Python library)!!2134 2135 \item If you need to do a check of control-C inside a loop, e.g.,2136 when constructing a matrix, put \code{_sig_check} instead of using2137 \code{_sig_on} then \code{_sig_off}.2138 \end{enumerate}2139 2140 \subsection{Introspection}2141 Users currently can't view the source code of functions2142 from the command line. This must change. Ideas for fixing2143 this problem.2144 2145 If we can find the module name where foo is defined,2146 then check if the appropriate foo.pyx file is available.2147 Load it. Find the function. Figure out which if it2148 is defined twice. Show all if we can't figure it out.2149 2150 If totally screwed and we can't even find the file, then2151 modify pyrex so it embeds extra info in the docstring.2152 Strip that out on displaying.2153 2154 %\section{Automatic Coercion to and From C Data Types}2155 %Integer literals2156 %type checking2157 %Etc.2158 %How to do explicit casts2159 2160 \subsection{Faster Iteration through a Python List or Tuple}2161 For example, define a function \code{mysum} in Cython as2162 follows. Note the trick of void2163 pointers to avoid Cython reference counting, etc:2164 2165 \begin{verbatim}2166 def mysum(x):2167 cdef object seq2168 cdef int len,i2169 cdef object item2170 cdef object total2171 total = 02172 cdef PyObject** X2173 X = FAST_SEQ_UNSAFE(x)2174 for i from 0 <=i<len:2175 item = <object>X[i]2176 total = total + item2177 return (total)2178 \end{verbatim}2179 The call to \code{FAST_SEQ_UNSAFE} gets direct very very2180 unsafe access to the underlying array of PyObject's.2181 2182 Somewhat surprisingly, this is actually {\em faster}2183 than Python's built in sum function:2184 %skip2185 \begin{verbatim}2186 sage: time sum(range(5*10^6))2187 CPU times: user 1.59 s, sys: 0.08 s, total: 1.67 s2188 12499997500000L2189 sage: time mysum(range(5*10^6))2190 CPU times: user 1.46 s, sys: 0.03 s, total: 1.49 s2191 12499997500000L2192 \end{verbatim}2193 2194 2195 In the next example, we illustrate the above idea by defining2196 several simplified versions of the \code{range} command.2197 2198 \begin{verbatim}%skip2199 %cython2200 2201 def f0(int n):2202 return eval('[%s for _ in xrange(%s)]'%(n, n))2203 2204 def f1(int n):2205 v = []2206 for i from 0 <= i < n:2207 v.append(i)2208 return v2209 2210 def f2(int n):2211 cdef int i2212 v = [None] * n2213 2214 for i from 0 <= i < n:2215 v[i] = i2216 return v2217 2218 def f3(int n):2219 cdef int i2220 w = [None] * n2221 cdef void** X2222 X = FAST_SEQ_UNSAFE(w)2223 for i from 0 <= i < n:2224 Py_DECREF(<PyObject*>X[i])2225 X[i] = PyInt_FromLong(i)2226 return w2227 2228 def f4(int n):2229 cdef int i2230 v = PyList_New(n)2231 for i from 0 <= i < n:2232 PyList_SET_ITEM(v, i, <object>PyInt_FromLong(i))2233 #WARNING -- maybe I need to incref the ref to the long?!2234 return v2235 \end{verbatim}2236 2237 The fourth one is the fastest.2238 2239 \subsection{getslice}2240 For fastest speed, use this declaration:2241 \begin{verbatim}2242 def __getslice__(self, Py_ssize_t i, Py_ssize_t j):2243 ...2244 \end{verbatim}2245 2246 %\section{The Python/C API}2247 % The {\tt object} type.2248 %\section{C Header Files}2249 %\section{Writing Your Code in a Mix of Cython and C/C++}2250 %\section{Splitting Implementations Across Multiple Files}2251 %\section{Wrapping C++ Code}2252 %Cython code compiles with C++ compiler. Using pyrexembed.2253 %\section{Other Optimization Tricks}2254 2255 %\section{Misc}2256 %\subsection{Class methods}2257 %In a cdef'd class this is not possible at all.2258 2259 \subsection{List Comprehensions and In-Place operators}2260 List comprehensions and in-place operators are fully supported in Cython.2261 (They are not supported in Pyrex, but Robert Bradshaw added support2262 for them to Cython.)2263 2264 \subsection{Cython: Dealing with weird build errors}2265 \begin{itemize}2266 \item {\bf Problem:} You get an error like this2267 \begin{verbatim}%skip2268 ...2269 lude/python2.5 -c sage/modules/complex_double_vector.c2270 -o build/temp.macosx-10.3-i386-2.5/sage/modules/complex_double_vector.o -w2271 sage/modules/complex_double_vector.c:366:2272 error: field '__pyx_base' has incomplete type2273 sage/modules/complex_double_vector.c:373:2274 error: field '__pyx_base' has incomplete type2275 error: command 'gcc' failed with exit status 12276 sage: There was an error installing modified sage library code.2277 \end{verbatim}2278 {\bf Solution:} In this case I added \code{cimport sage.structure.element} to \code{complex_double_vector.pxd}2279 and the problem disappeared. In general, look at the generated C file to see what type is incomplete (i.e.,2280 not properly declared), then add an explicit cimport to your pxd file (or top of your pyx file if you2281 don't have a pxd file).2282 2283 \item {\bf Problem:} Errors such as "XXX does not appear to be the correct type object".\\2284 {\bf Solution:} This can happen when the build system doesn't properly detect a dependency2285 and regenerate the corresponding files. If you know which .pyx file should be recompiled,2286 just touch it. If you don't, you can always type \code{sage -ba} to brutally2287 rebuild everything, which will take about 3-5 minutes.2288 2289 \end{itemize}2290 2291 %\section{To Add}2292 %\begin{verbatim}2293 %Parsifal Herzog wrote:2294 % > cdef va_list ap <--- line 1122295 % > cdef voidptr oi2296 % > va_start(ap, o1)2297 % > oi = va_arg(ap, voidptr)2298 % > while oi != 0:2299 % > l.append(<object>oi)2300 % > oi = va_arg(ap, voidptr)2301 % > va_end(ap)2302 2303 % Here is an example how to use va_args:2304 2305 % http://codespeak.net/svn/lxml/trunk/src/lxml/cstd.pxd2306 2307 % The macros are defined at the the top of this file:2308 2309 % http://codespeak.net/svn/lxml/trunk/src/lxml/etree_defs.h2310 2311 % Stefan2312 2313 2314 % _______________________________________________2315 % Pyrex mailing list2316 % Pyrex@lists.copyleft.no2317 % http://lists.copyleft.no/mailman/listinfo/pyrex2318 % \end{verbatim}2319 2320 % \begin{verbatim}2321 % > Ok2322 % >2323 % > about my casting question. I should have declared2324 % >2325 % > cdef int c_length2326 % >2327 % > in which case the code works correctly. However in one place I had2328 % > forgot the int and just typed2329 % >2330 % > cdef c_length2331 % >2332 % > and this led to the example I sent. What type of object is created if2333 % > you declare with cdef but forget the type.2334 2335 % The default is a Python object. It's equivalent to2336 % cdef object c_length.2337 2338 % William2339 % \end{verbatim}2340 2341 \section{C++ Nuances}2342 (This section was written by Joel B. Mohler. Many of the tricks come from2343 Martin Albrecht and David Harvey.)2344 2345 Cython can generate C++ for wrapping of C++ libraries. There are just a2346 few things to keep in mind though. Cython doesn't know specifically about2347 C++ features like function and operator overloading. Therefore, your library2348 must have unique function names or you must alias them. Other issues2349 which arise have to do with functions taking references and pointer2350 dereferencing since C does not support either of these.2351 2352 You can see real live examples for many of these concepts in the NTL2353 wrapper classes. These are in the \SAGE tree at \code{sage/libs/ntl/}.2354 2355 \subsection{Aliases for Overloading (or other purposes)}2356 2357 Cython doesn't support overloaded functions, but C++ libraries readily use2358 this feature. Cython does provide a very simple way to alias these functions2359 which we illustrate with an example borrowed from the NTL wrapper code.2360 2361 \begin{verbatim}2362 cdef extern from "ntl_wrap.h":2363 # really, this is from NTL/ZZ.h2364 ctypedef struct ZZ_c "struct ZZ":2365 pass2366 void ZZ_add "add"( ZZ_c x, ZZ_c a, ZZ_c b)2367 2368 #### ZZ_p_c2369 ctypedef struct ZZ_p_c "struct ZZ_p":2370 pass2371 void ZZ_p_add "add"( ZZ_p_c x, ZZ_p_c a, ZZ_p_c b)2372 \end{verbatim}2373 2374 Notice that NTL supports the \code{add} function for all of its many types.2375 We rename them for use in Cython as \code{<Type>_add} where \code{<Type>}2376 is the NTL type name of the objects being added.2377 2378 Another type of aliasing in that example is the structure \code{ZZ} is called2379 \code{ZZ_c}. We did this simply to alleviate confusion with the \SAGE integer2380 class commonly known as \code{ZZ}. This trick is also useful if you have a2381 C++ library that uses namespaces. (Actually NTL does use namespaces,2382 but we were lazy and instead inserted a \code{using namespaces NTL;} in2383 a header file).2384 2385 \subsection{Pointers and References}2386 2387 Given a pointer to a C/C++ type, Cython does not support the use of '*' to2388 dereference this pointer. Here's a way around that:2389 \begin{verbatim}2390 cdef Foo *t2391 t = <something>2392 function_that_takes_a_reference(t[0])2393 \end{verbatim}2394 In C, \code{t[0]} is the first element of the array pointed to by \code{t}. Remember2395 that C arrays are simply pointers. Note that this trick is rarely necessary in C2396 since most C code operates by passing around pointers to structures. They are2397 not ordinarily dereferenced.2398 2399 \subsection{Embedding C++ objects in Cython extension structures}2400 2401 Suppose that you have a C++ class (I'll call it Foo) which you want to wrap for2402 access from python. There would be two basic ways to go about managing2403 this structure:2404 2405 \begin{itemize}2406 \item Initialize a point to the object in the \code{__new__} function in my2407 python/Cython object and delete in the \code{__dealloc__}. This method2408 is pretty easy to understand from a C++ perspective.2409 \item Declare the object as member of the Cython cdef'ed class. This method2410 is not quite so straightforward, but it saves you a memory allocation. This might2411 be a decent speed gain for elements of some ring where an object is allocated2412 for every arithmetic operation.2413 \end{itemize}2414 2415 We let the details of the first option as an exercise to the reader and provide a2416 more elaborate description for the second option. Use code like the following2417 to embed the object in the Cython extension class structure:2418 \begin{verbatim}2419 cdef class FooWrapper:2420 cdef Foo x2421 \end{verbatim}2422 2423 This may appear to work fine because many C++ constructors don't do a whole2424 lot of interesting work and Cython zeroes out the memory in these structures.2425 However, there is a problem since neither the constructor nor destructor are called2426 for Foo since the memory is allocated with malloc (or PyMalloc or something).2427 So, you need to call the constructor and destructor manually. The rest of this2428 section provides a simple way to do so.2429 2430 The main trick is that we need to call the constructor of Foo, but the memory2431 is already allocated. An important nuance of the C++ \code{new} operator is2432 described here:2433 \url{http://publib.boulder.ibm.com/infocenter/macxhelp/v6v81/index.jsp?topic=/com.ibm.vacpp6m.doc/language/ref/clrc05cplr199.htm}.2434 There is a C++ header file in the \code{sage_c_lib} which defines some templated2435 functions to make writing wrappers for C++ classes more convenient. You can2436 make the following declarations in a pxd or pxi file for use from Cython.2437 \begin{verbatim}2438 cdef extern from "Foo.h":2439 ctypedef struct Foo:2440 pass2441 2442 cdef extern from "ccobject.h":2443 # Some non-allocating versions2444 Foo* Foo_construct "Construct<Foo>"(void *mem)2445 void Foo_destruct "Destruct<Foo>"(Foo *mem)2446 2447 # some allocating versions for completeness2448 Foo* Foo_new "New<Foo>"()2449 void Foo_delete "Delete<Foo>"(Foo *mem)2450 \end{verbatim}2451 2452 Now, we need to call these functions we defined from our Cython2453 \code{__new__} and \code{__dealloc__} functions.2454 \begin{verbatim}2455 cdef class FooWrapper:2456 def __init__(self, ...):2457 pass2458 2459 # Note that the parameters to __new__ must be identical to __init__2460 # This is due to some Cython vagary2461 def __new__(self, ...):2462 Foo_construct(&self.x)2463 2464 def __dealloc__(self):2465 Foo_destruct(&self.x)2466 \end{verbatim}2467 2468 \chapter{\SAGE{} Interfaces}2469 This chapter is about SAGE interfaces, both2470 the C/C++ level interfaces, and the pseduo-tty interfaces,2471 in \SAGE programming.2472 1299 2473 1300 \section{The PARI C-library Interface} 1301 2474 1302 (This chapter was written by Martin Albrecht.) 2475 1303 2476 1304 Here is the step-by-step guide to adding a new PARI functions to \SAGE. 2477 1305 We use the Frobenius form of a matrix as an example. 2478 1306 2479 1307 The heavy lifting for matrices over integers is implemented using the 2480 PARI library. To compute the frobenius form in PARI the1308 PARI library. To compute the Frobenius form in PARI the 2481 1309 \code{matfrobenius} function is used. 2482 1310 2483 1311 There are two ways to interact with the PARI library from \SAGE: The gp 2484 interface uses the gp interpreter and the PARI interface uses direct calls to 2485 the PARI C functions and is the preferred way as it is much faster. 1312 interface uses the gp interpreter, and the PARI interface uses direct calls to 1313 the PARI C functions --- this is the preferred way as it is much 1314 faster. Thus this section focuses on using PARI. 2486 1315 2487 So we need to add a new method to the gen class which is the abstract 2488 representation of all PARI library objects. That means that if we add a 2489 method to this class every PARI object regardless if it is a number, 2490 polynomial or matrix will have our new method. So you can do 2491 \code{pari(1).matfrobenius()} but you will receive a PariError in this case. 1316 So we will add a new method to the gen class: this is the abstract 1317 representation of all PARI library objects. That means that once we add a 1318 method to this class, every PARI object, whether it is a number, 1319 polynomial or matrix, will have our new method. So you can do 1320 \code{pari(1).matfrobenius()}, but since PARI wants to apply 1321 \code{matfrobenius} to matrices, not numbers, you will receive a 1322 PariError in this case. 2492 1323 2493 The gen class is defined in sage/libs/pari/gen.pyx where we add the method 2494 \code{matfrobenius}: 1324 The gen class is defined in 1325 \code{SAGE_ROOT/devel/sage/sage/libs/pari/gen.pyx}, and this is where we add 1326 the method \code{matfrobenius}: 2495 1327 2496 1328 \begin{verbatim} 2497 1329 … … 2507 1339 return self.new_gen(matfrobenius(self.g, flag)) 2508 1340 \end{verbatim} 2509 1341 2510 The \ verb+_sig_on+ statement is some magic to prevent SIGSEGVs from the pariC1342 The \code{_sig_on} statement is some magic to prevent SIGSEGVs from the PARI C 2511 1343 library to crash the \SAGE interpreter by catching these signals. The 2512 \ verb+self.new_gen()+call constructs a new SAGE-python-gen object from a given1344 \code{self.new_gen()} call constructs a new SAGE-python-gen object from a given 2513 1345 pari-C-gen where the pari-C-gen is stored as the SAGE-python-gen.g attribute. 2514 The matfrobenius call is just a call to the pariC library function2515 "matfrobenius"with the appropriate parameters.1346 The \code{matfrobenius} call is just a call to the PARI C library function 1347 \code{matfrobenius} with the appropriate parameters. 2516 1348 2517 The information which function to call and how to call it can be retrieved2518 from the PARI user s manual (note: \SAGE includes the development version of2519 PARI so check the development version's user manual). Looking for1349 The information about which function to call and how to call it can be retrieved 1350 from the PARI user's manual (note: \SAGE includes the development version of 1351 PARI, so check that version of the user's manual). Looking for 2520 1352 \code{matfrobenius} you can find: 2521 \verb+"The library syntax is matfrobenius(M,flag)"+ 1353 \verb+"The library syntax is matfrobenius(M,flag)"+. 2522 1354 2523 Please not that the pari C function may have a different name than gp function 2524 (see e.g. \code{mathnf}), so always check with the manual. 1355 In case you are familiar with gp, 1356 please note that the PARI C function may have a different name than 1357 the corresponding gp function 1358 (for example, see \code{mathnf}), so always check the manual. 2525 1359 2526 At this point we are done with the PARI interface and mayadd some more1360 At this point we are done with the PARI interface and can add some more 2527 1361 interfaces around it for convenience: 2528 2529 First we add a functional representation of the method to 2530 sage/libs/pari/functional.py so we can do: 2531 \verb+matfrobenius(<some.pari.gen>)+ additionally to 2532 \verb+<some.pari.gen>.matfrobenius()+. 2533 1362 first we add a functional representation of the method to 1363 \code{SAGE_ROOT/devel/sage/sage/libs/pari/functional.py} 1364 so we can do 1365 \code{matfrobenius(<some.pari.gen>)} as well as 1366 \code{<some.pari.gen>.matfrobenius()}: 2534 1367 \begin{verbatim} 2535 1368 def matfrobenius(self): return pari(self).matfrobenius() 2536 1369 \end{verbatim} 2537 1370 2538 Then we also add a frobenius(flag) method to the matrix_integer class where we 2539 call the \code{matfrobenius()} method on the \verb+_pari_()+ object associated with the 2540 matrix after doing some sanity checking. Then we need to convert the pari gen 2541 to some meaningful \SAGE objects depending on the return value as described in 2542 the PARI user's manual. 2543 1371 Then we also add a \code{frobenius(flag)} method to the 1372 \code{matrix_integer} class where we call the \code{matfrobenius()} 1373 method on the PARI object associated to the matrix after doing some 1374 sanity checking. Then we convert output from PARI to \SAGE objects: 2544 1375 \begin{verbatim} 2545 1376 def frobenius(self,flag=0): 2546 1377 """ 2547 Return the Frobenius form of this matrix. 1378 If flag is 0 (the default value), return the Frobenius 1379 form of this matrix. 2548 1380 If flag is 1, return only the elementary divisors. 2549 If flag is 2, return a two-component svector [F,B]2550 where F is the Frobenius form and B is the basis change2551 so that M=B^-1*F*B.1381 If flag is 2, return a two-component vector [F,B] 1382 where F is the Frobenius form and B is the basis change 1383 so that M=B^-1*F*B. 2552 1384 2553 1385 INPUT: 2554 1386 flag -- 0,1 or 2 as described above … … 2579 1411 elif flag==1: 2580 1412 r = polynomial_ring.PolynomialRing(self.base_ring()) 2581 1413 #BUG: this should be handled in PolynomialRing not here 2582 return [eval(str(x).replace("^","**"),{},r.gens_dict()) for x in v.python_list()] 1414 return [eval(str(x).replace("^","**"),{},r.gens_dict()) 1415 for x in v.python_list()] 2583 1416 elif flag==2: 2584 F = matrix_space.MatrixSpace(rational_field.RationalField(),self.nrows())(v[0].python()) 2585 B = matrix_space.MatrixSpace(rational_field.RationalField(),self.nrows())(v[1].python()) 1417 F = matrix_space.MatrixSpace(rational_field.RationalField(), 1418 self.nrows())(v[0].python()) 1419 B = matrix_space.MatrixSpace(rational_field.RationalField(), 1420 self.nrows())(v[1].python()) 2586 1421 return F,B 2587 2588 1422 \end{verbatim} 2589 1423 2590 Then we add some examples, wait for make test to complete without errors and2591 commit.2592 1424 2593 1425 \section{GAP} 2594 1426 … … 2596 1428 2597 1429 Wrapping a GAP function in \SAGE is a matter of writing a program in 2598 1430 Python which uses the pexpect interface to pipe various commands to 2599 GAP and read back the input into \SAGE. This can range from easy to1431 GAP and read back the input into \SAGE. This is sometimes easy, sometimes 2600 1432 hard. 2601 1433 2602 For example, suppose we want to make a wrapper for computation of the1434 For example, suppose we want to make a wrapper for the computation of the 2603 1435 Cartan matrix of a simple Lie algebra. The Cartan matrix of $G_2$ is 2604 1436 available in GAP using the commands 2605 1437 %skip … … 2613 1445 (Incidentally, most of the GAP Lie algebra implementation was written by 2614 1446 Thomas Breuer, Willem de Graaf and Craig Struble.) 2615 1447 2616 In \SAGE, one can simply type1448 In \SAGE, one can access these commands by typing 2617 1449 2618 1450 \begin{verbatim} 2619 1451 sage: L = gap.SimpleLieAlgebra('"G"', 2, 'Rationals'); L … … 2626 1458 \end{verbatim} 2627 1459 Note the \code{'"G"'} which is evaluated in Gap as the string \code{"G"}. 2628 1460 2629 Using this example, the purpose of this section is 2630 to show how one might write a Pyhon/\SAGE program 2631 who's input is, say, \code{('G',2)} and who's output 2632 is the matrix above (but as a \SAGE Matrix - see 2633 \code{matrix.py} in \code{sage/matrix/matrix.py}). 1461 The purpose of this section is to use this example 1462 to show how one might write a Python/\SAGE program 1463 whose input is, say, \code{('G',2)} and whose output 1464 is the matrix above (but as a \SAGE Matrix --- see 1465 the code in the directory \code{SAGE_ROOT/devel/sage/sage/matrix/} and the 1466 corresponding parts of the \sage reference manual). 2634 1467 2635 1468 First, the input must be converted into strings consisting of legal 2636 1469 GAP commands. Then the GAP output, which is also a string, must be … … 2662 1495 2663 1496 L = gap.SimpleLieAlgebra('"%s"'%type, rank, 'Rationals') 2664 1497 R = L.RootSystem() 2665 sM = R.CartanMatrix()2666 ans = eval(str(sM))2667 MS = MatrixSpace(ZZ, rank)2668 return MS(ans)2669 \end{verbatim}2670 The output {\tt ans} is a Python list. The last two lines convert that2671 list to a \SAGE class object Matrix instance.2672 2673 Alternatively, one could code the body of the above function in2674 a more pythonic way as follows:2675 %skip2676 \begin{verbatim}2677 L = gap.new(SimpleLieAlgebra("%s", %s, Rationals);'%(type, rank))2678 R = L.RootSystem()2679 1498 sM = R.CartanMatrix() 2680 1499 ans = eval(str(sM)) 2681 1500 MS = MatrixSpace(QQ, rank) 2682 1501 return MS(ans) 2683 1502 \end{verbatim} 1503 The output \code{ans} is a Python list. The last two lines convert that 1504 list to an instance of the \sage class \code{Matrix}. 1505 1506 Alternatively, one could replace the first line of the above function 1507 with this: 1508 %skip 1509 \begin{verbatim} 1510 L = gap.new(SimpleLieAlgebra("%s", %s, Rationals);'%(type, rank)) 1511 \end{verbatim} 2684 1512 2685 1513 Defining ``easy'' and ``hard'' is subjective, but here is one 2686 definition: an exampleis ``easy'' if there is already a corresponding2687 class objectin Python or \SAGE for the output data type of the GAP1514 definition: wrapping a GAP function is ``easy'' if there is already a corresponding 1515 class in Python or \SAGE for the output data type of the GAP 2688 1516 function you are trying to wrap. For example, wrapping any GUAVA 2689 1517 (GAP's error-correcting codes package) function is ``easy'' since 2690 1518 error-correcting codes are vector spaces over finite fields and GUAVA … … 2707 1535 \item[(e)] 2708 1536 integers. 2709 1537 \end{itemize} 2710 \SAGE already has a class objectfor each of these.1538 \SAGE already has classes for each of these. 2711 1539 2712 1540 A ``hard'' example is left as an exercise! Here are a few ideas. 2713 1541 2714 1542 \begin{itemize} 2715 1543 2716 \item Write a wrapper for GAP's {\ttFreeLieAlgebra} function (or,1544 \item Write a wrapper for GAP's \code{FreeLieAlgebra} function (or, 2717 1545 more generally, all the finitely presented Lie algebra fuunctions in 2718 1546 GAP). This would require creating new Python objects. 2719 1547 2720 \item Write a wrapper for GAP's {\ttFreeGroup} function (or, more1548 \item Write a wrapper for GAP's \code{FreeGroup} function (or, more 2721 1549 generally, all the finitely presented groups fuunctions in GAP). 2722 1550 This would require writing some new Python objects. 2723 1551 … … 2728 1556 \end{itemize} 2729 1557 2730 1558 \section{Singular} 1559 2731 1560 (The first version of this chapter was written by David Joyner.) 2732 1561 2733 Using Singular functions from \SAGE is not much different conceptually than using2734 GAP functions from \SAGE. This can range from very easy to hard, depending on how 2735 much of the data structure of the output of the Singular function is already 2736 present in \SAGE. 1562 Using Singular functions from \SAGE is not much different conceptually 1563 than using GAP functions from \SAGE. As with GAP, this can range from 1564 easy to hard, depending on how much of the data structure of the 1565 output of the Singular function is already present in \SAGE. 2737 1566 2738 First, some terminology. For us, a \emph{curve} $X$ over a finite field $F$ is a 1567 First, some terminology. For us, a \emph{curve} $X$ over a finite field $F$ is an 2739 1568 equation of the form $f(x,y)=0$, where $f\in F[x,y]$ is a polynomial. It may or 2740 1569 may not be singular. A \emph{place of degree} $d$ is a Galois orbit of $d$ 2741 1570 points in $X(E)$, where $E/F$ is degree $d$. For example, a place of degree $1$ … … 2951 1780 % } 2952 1781 2953 1782 2954 >From looking at the output, notice that our wrapper program will need to 2955 parse the string represented by $L$ above. 2956 2957 Let us write a separate program to do just that. This requires figuring out how 1783 From looking at the output, notice that our wrapper program will need to 1784 parse the string represented by $L$ above, so 1785 let us write a separate program to do just that. This requires figuring out how 2958 1786 to determine where the coordinates of the points are placed in the string L. 2959 1787 Python has some very useful string manipulation commands to do just that. 2960 1788 … … 3012 1840 3013 1841 3014 1842 Now it is an easy matter to put these ingredients together into a \SAGE function 3015 which takes as input a triple $(f,F,d)$: finite field $F$ \emph{of prime order}, 3016 a polynomial $f$ in $F[x,y]$ defining $X:\ f(x,y)=0$ (note the variables $x,y$ 3017 must be used), and the degree $d$. The output is the number of points of places 3018 in $X$ of degree $d=1$ over $F$. At the moment, their is no ``translation'' 1843 which takes as input a triple $(f,F,d)$: 1844 a polynomial $f$ in $F[x,y]$ defining $X:\ f(x,y)=0$ (note that the variables $x,y$ 1845 must be used), 1846 a finite field $F$ \emph{of prime order}, 1847 and the degree $d$. The output is the number of places 1848 in $X$ of degree $d=1$ over $F$. At the moment, there is no ``translation'' 3019 1849 between elements of $GF(p^d)$ in Singular and \SAGE unless $d=1$. So, for this 3020 1850 reason, we restrict ourselves to points of degree one. 3021 1851 … … 3052 1882 return points_parser(L,F) 3053 1883 3054 1884 \end{verbatim} 3055 The triple quoted string documentation is important! It not only helps users 3056 understand your function's use and syntax but also, if the function is included 3057 in \SAGE, will be reproduced in the printed and online documentation, and 3058 \emph{automatically tested} (even the output) before each \SAGE release. 3059 3060 Note also that the ordering returned by this \SAGE function is exactly the same 1885 Note that the ordering returned by this \SAGE function is exactly the same 3061 1886 as the ordering in the Singular variable \code{POINTS}. 3062 1887 3063 1888 %\comment{ … … 3076 1901 %number_of_places(f,F,d) 3077 1902 %} 3078 1903 3079 Some examples:1904 One more example (in addition to the one in the docstring): 3080 1905 3081 1906 %skip 3082 1907 \begin{verbatim} 3083 sage: F = GF(5)3084 sage: R = MPolynomialRing(F,2,names = ["x","y"])3085 sage: x,y = R.gens()3086 sage: f = y^2-x^9-x3087 sage: places_on_curve(f,F)3088 ((0, 1, 0), (3, 4, 1), (0, 0, 1), (2, 3, 1), (3, 1, 1), (2, 2, 1))3089 3090 1908 sage: F = GF(2) 3091 1909 sage: R = MPolynomialRing(F,2,names = ["x","y"]) 3092 1910 sage: x,y = R.gens() … … 3095 1913 ((0, 1, 0), (1, 0, 0), (0, 0, 1)) 3096 1914 \end{verbatim} 3097 1915 3098 \section{Another Approach} 3099 There is also a more python-like interface to Singular. Using this 1916 \section{Singular: Another Approach} 1917 1918 There is also a more Python-like interface to Singular. Using this 3100 1919 the code is much simpler, as illustrated below. First we demonstrate 3101 1920 computing the places on a curve in a particular case. 3102 1921 … … 3116 1935 sage: [(L[i][1], L[i][2], L[i][3]) for i in range(1,7)] 3117 1936 [(0, 1, 0), (-2, 1, 1), (0, 0, 1), (2, 2, 1), (-2, -1, 1), (2, -2, 1)] 3118 1937 \end{verbatim} 3119 The \code{name} method of a Singular object returns the name3120 of that object in the Singular interpreter, so that it can3121 be used as input to a Singular function.3122 1938 3123 Next we implement the general function ( we omit the docstring, which1939 Next we implement the general function (for brevity we omit the docstring, which 3124 1940 is the same as above). Note that the \code{point_parser} function 3125 1941 is not required. 3126 1942 … … 3149 1965 used only the barest minimum of that interface. 3150 1966 3151 1967 \section{Creating a new Pseudo-tty Interface} 3152 \SAGE pseudo-tty interfaces can be created that allow \SAGE to 3153 work with an almost completely arbitrary command line program, 3154 and don't require any modification or extensions to the 3155 command line program. They are also {\em surprisingly} fast 1968 1969 You can create \SAGE pseudo-tty interfaces that allow \SAGE to 1970 work with almost any command-line program, 1971 and which don't require any modification or extensions to that 1972 program. They are also {\em surprisingly} fast 3156 1973 and flexible (given how they work!), because all IO is 3157 1974 buffered, and because interaction between \sage and the 3158 the command line program can be non-blocking (asynchronous), 3159 using the \code{_send}, \code{_so_far}, and \code{_get} 3160 methods. 1975 command line program can be non-blocking (asynchronous); this is 1976 because they all derive from the \sage class \code{Expect}, which 1977 handles the communication between \sage and the external process. 3161 1978 3162 % To create a new interface to a computer algebra system, 3163 % do the following: 3164 % \begin{enumerate} 3165 % \item Copy 1979 For example, here is part of the file 1980 \code{SAGE_ROOT/devel/sage/sage/interfaces/octave.py}, which defines 1981 an interface between \sage and Octave, an open-source program 1982 for doing numerical computations, among other things. 1983 %skip 1984 \begin{verbatim} 1985 import os 1986 from expect import Expect, ExpectElement 3166 1987 3167 % \section{Projects} 3168 % For the interested reader, here are a few ideas for some other wrappers, 3169 % which are not yet implemented in \SAGE. 1988 class Octave(Expect): 1989 \end{verbatim} 1990 The first two lines import the library \code{os}, which contains 1991 operating system routines, and also class \code{Expect}, which is the 1992 basic class for interfaces. The third line defines the class 1993 \code{Octave}: it derives from \code{Expect}. After this comes a 1994 docstring, which we omit here -- see the file for details. Next comes: 1995 %skip 1996 \begin{verbatim} 1997 def __init__(self, maxread=100, script_subdirectory="", logfile=None, 1998 server=None, server_tmpdir=None): 1999 Expect.__init__(self, 2000 name = 'octave', 2001 prompt = '>', 2002 command = "octave --no-line-editing --silent", 2003 maxread = maxread, 2004 server = server, 2005 server_tmpdir = server_tmpdir, 2006 script_subdirectory = script_subdirectory, 2007 restart_on_ctrlc = False, 2008 verbose_start = False, 2009 logfile = logfile, 2010 eval_using_file_cutoff=100) 2011 \end{verbatim} 2012 This uses the class \code{Expect} to set up the Octave interface. 2013 %skip 2014 \begin{verbatim} 2015 def set(self, var, value): 2016 """ 2017 Set the variable var to the given value. 2018 """ 2019 cmd = '%s=%s;'%(var,value) 2020 out = self.eval(cmd) 2021 if out.find("error") != -1: 2022 raise TypeError, "Error executing code in Octave\nCODE:\n\t%s\nOctave ERROR:\n\t%s"%(cmd, out) 3170 2023 3171 % \begin{itemize} 2024 def get(self, var): 2025 """ 2026 Get the value of the variable var. 2027 """ 2028 s = self.eval('%s'%var) 2029 i = s.find('=') 2030 return s[i+1:] 3172 2031 3173 % \item 3174 % Write a \SAGE function, possibly containing Singular code, 3175 % which ``translates'' the Singular generator 3176 % \code{a} of $\GF(p^d)$ to a \SAGE generator of $\GF(p^d)$. 2032 def console(self): 2033 octave_console() 2034 \end{verbatim} 2035 These let users type \code{octave.set('x', 3)}, after which 2036 \code{octave.get('x')} returns \code{' 3'}. Running 2037 \code{octave.console()} dumps the user into Octave interactive shell. 2038 %skip 2039 \begin{verbatim} 2040 def solve_linear_system(self, A, b): 2041 """ 2042 Use octave to compute a solution x to A*x = b, as a list. 3177 2043 3178 % \item 3179 % Write a wrapper for Singular's 3180 % \code{BrillNoether} function to compute bases of Riemann-Roch spaces. 3181 % This would not require writing some new Python objects. 2044 INPUT: 2045 A -- mxn matrix A with entries in QQ or RR 2046 b -- m-vector b entries in QQ or RR (resp) 3182 2047 3183 % \item 3184 % Write a wrapper for Singular's 3185 % \code{groebner} function to compute Groebner bases. 3186 % This would not require writing some new Python objects. 2048 OUTPUT: 2049 An list x (if it exists) which solves M*x = b 3187 2050 3188 % \item 3189 % Write a wrapper for Singular's method for 3190 % ``Solving systems of polynomial equations'' in 3191 % \url{http://www.singular.uni-kl.de/Manual/3-0-0/sing_502.htm\#SEC554} 2051 EXAMPLES: 2052 sage: M33 = MatrixSpace(QQ,3,3) 2053 sage: A = M33([1,2,3,4,5,6,7,8,0]) 2054 sage: V3 = VectorSpace(QQ,3) 2055 sage: b = V3([1,2,3]) 2056 sage: octave.solve_linear_system(A,b) # requires optional octave 2057 [-0.33333299999999999, 0.66666700000000001, -3.5236600000000002e-18] 3192 2058 3193 % \item 3194 % Write a wrapper for Singular's method for constructing and decoding 3195 % ``AG codes'' in 3196 % \url{http://www.singular.uni-kl.de/Manual/3-0-0/sing_503.htm\#SEC555} 3197 % \end{itemize} 2059 AUTHOR: David Joyner and William Stein 2060 """ 2061 m = A.nrows() 2062 n = A.ncols() 2063 if m != len(b): 2064 raise ValueError, "dimensions of A and b must be compatible" 2065 from sage.matrix.all import MatrixSpace 2066 from sage.rings.all import QQ 2067 MS = MatrixSpace(QQ,m,1) 2068 b = MS(list(b)) # converted b to a "column vector" 2069 sA = self.sage2octave_matrix_string(A) 2070 sb = self.sage2octave_matrix_string(b) 2071 self.eval("a = " + sA ) 2072 self.eval("b = " + sb ) 2073 soln = octave.eval("c = a \\ b") 2074 soln = soln.replace("\n\n ","[") 2075 soln = soln.replace("\n\n","]") 2076 soln = soln.replace("\n",",") 2077 sol = soln[3:] 2078 return eval(sol) 2079 \end{verbatim} 2080 This code defines the method \code{solve_linear_system}, which works 2081 as documented. 3198 2082 2083 These are only excerpts from \code{octave.py}; check that file for 2084 more definitions and examples. Look at other files in the directory 2085 \code{SAGE_ROOT/devel/sage/sage/interfaces/} for examples of 2086 interfaces to other software packages. 3199 2087 3200 \chapter{Mercurial: The \sage Source Control System} 2088 2089 2090 \chapter{The \SAGE Manuals} 2091 \label{ch:manuals} 2092 2093 This chapter describes how to modify the \SAGE manuals. 2094 \sage's manuals are written in \Latex, and to edit them, you just need 2095 to edit the appropriate file. 2096 2097 If, for example, you want to change the \sage tutorial, then you 2098 should start by modifying the file 2099 \code{SAGE_ROOT/devel/doc/tut/tut.tex}. Then to build a dvi or pdf file 2100 with your changes, type \code{./build_dvi} or 2101 \code{./build_pdf} in the \code{devel/doc/tut} directory. You'll get 2102 a file \code{tut.dvi} or \code{tut.pdf}, which you should inspect. 2103 You should also run \code{sage -t tut.tex} to test all of the examples 2104 -- see Section~\ref{ch:testing}. 2105 You can build an HTML 2106 version of the tutorial by typing \code{make tut} in the 2107 \code{devel/doc} directory. Once you've done this, you can access the 2108 new HTML version from the notebook interface to \sage by clicking the 2109 \code{Help} link. 2110 2111 Finally, you might want to share your changes with the \sage 2112 community. To do this, use Mercurial (see Chapter~\ref{ch:mercurial}) 2113 to produce patch files, and submit them to the \sage trac server. 2114 2115 The \sage manuals and the corresponding files to edit: 2116 \begin{itemize} 2117 \item The \sage tutorial: 2118 \code{SAGE_ROOT/devel/doc/tut/tut.tex} 2119 \item The \sage programming guide: 2120 \code{SAGE_ROOT/devel/doc/prog/prog.tex} 2121 \item Constructions in \sage: 2122 \code{SAGE_ROOT/devel/doc/const/const.tex} 2123 \item The \sage installation guide: 2124 \code{SAGE_ROOT/devel/doc/inst/inst.tex} 2125 \item The \sage reference manual: some of this is contained in the file 2126 \code{SAGE_ROOT/devel/doc/ref/ref.tex}, but most of it is 2127 automatically generated from the \sage source code. See the file 2128 \code{SAGE_ROOT/devel/doc/ref/README.txt} for information about how to 2129 edit the reference manual. 2130 \end{itemize} 2131 2132 2133 2134 2135 2136 \part{Disseminating Code for \sage} 2137 2138 \chapter{Introduction} 2139 2140 Whether you've developed some new code for \sage or just have a simple 2141 bug fix, you need to know how to communicate what you've done to other 2142 \sage users. This part of the guide discusses this issue. Here are 2143 some of the available avenues of communication and tools to aid in 2144 that communication: 2145 2146 \begin{itemize} 2147 \item the Google group \code{sage-support}, at 2148 \url{http://groups.google.com/group/sage-support}. According to the 2149 group's official description, ``This group gives help and support to 2150 those who have problems with \sage (installation, syntax, etc).'' 2151 \item the Google group \code{sage-devel}, at 2152 \url{http://groups.google.com/group/sage-devel}. According to the 2153 group's description, ``This list is for \sage development and is about 2154 programming, design and technical issues.'' This is a great place to 2155 post questions about whether certain behavior is a bug, or whether a 2156 certain feature ought to be implemented (or how it ought to be 2157 implemented), or similar issues. 2158 \item Mercurial: this is the source control system that is included 2159 with \SAGE. Use this to produce patches for \sage. See 2160 Chapter~\ref{ch:mercurial} for a full discussion of Mercurial. 2161 \item the \sage trac server, at 2162 \url{http://trac.sagemath.org/sage_trac/}: this is where you should 2163 post bugs, patches for bugs, additions to the \sage library, etc. See 2164 Chapter~\ref{ch:trac} for more information. 2165 \end{itemize} 2166 2167 2168 2169 \section{Inclusion Procedure for New \sage Code} 2170 2171 For code to become part of \sage's core, it must meet the 2172 following requirements: 2173 2174 \begin{itemize} 2175 \item \textbf{License}. The license must be a GPL version 2+ 2176 compatible license. (This will be publicly revisited around Jan 15, 2177 2009.) 2178 \item \textbf{Build Support}. 2179 The code must build on our supported architectures and compilers 2180 (and intended port targets): 2181 \begin{itemize} 2182 \item Linux: x86, x86_64, Itanium, ppc, ppc64, Sparc (gcc 3.4-4.3) 2183 \item Apple Mac OS X: ppc, ppc64, x86, x86-64 (Xcode 2.5+) 2184 \item Microsoft Windows: x86, x86_64 MSVC 2005/Intel Fortran (MinGW or 2185 Cygwin support is insufficient!) 2186 \item Solaris 10: Sparc, x86, x86_64 (Sun Forte 12) 2187 \end{itemize} 2188 Remarks: 2189 \begin{itemize} 2190 \item Some \sage developers are willing to help you port to OSX, 2191 Solaris and Windows. But this is no guarantee and you or your project 2192 are expected to do the heavy lifting and also support those ports 2193 upstream if there is no \sage developer who is willing to share the 2194 burden. 2195 \end{itemize} 2196 Potential future ports include FreeBSD (x86, x86-64), OpenBSD (x86, 2197 x86-64), HPUX (Itanium), AIX (PPC64), and ARM (OSX). 2198 \item \textbf{Quality}. The code should be ``better'' than any other 2199 available code (that passes the two above criteria), and the authors 2200 need to justify this. The comparison should be made to both Python and 2201 other software. Criteria in passing the quality test include: 2202 \begin{itemize} 2203 \item Speed 2204 \item Documentation 2205 \item Usability 2206 \item Memory leaks 2207 \item Maintainable 2208 \item Reasonable build time, size, dependencies 2209 \end{itemize} 2210 \item \textbf{Refereeing}. The code must be refereed, as discussed in 2211 Chapter~\ref{ch:trac}. 2212 \end{itemize} 2213 2214 2215 2216 2217 \chapter{Producing Patches with Mercurial} 3201 2218 \label{ch:mercurial} 3202 2219 3203 Mercurial is the official source control system that is included with 3204 \SAGE. Mercurial is robust and works well even with huge data sets. 3205 It's a solid piece of quality software. 3206 3207 {\it All} the Mercurial repositories related to \SAGE are included 3208 with \SAGE by default. Thus the complete change history and setup 3209 for doing development is available in your copy of \SAGE by 3210 default. It's not something that has to be confusingly installed 3211 later. \SAGE is designed to be extremely developer friendly. 2220 If you are editing or adding to \sage's core library, you will 2221 probably want to share your changes with other users. Mercurial is 2222 the tool to do this. Mercurial is the source control system that is 2223 included with \SAGE. This chapter provides an overview of how to use 2224 Mercurial with \sage; see \url{http://www.selenic.com/mercurial/} for 2225 full documentation on Mercurial. 3212 2226 3213 \SAGE includes by default these Mercurial repositories: 2227 All of the Mercurial repositories related to \SAGE are included 2228 with \SAGE. Thus the complete change history and setup 2229 for doing development is available in your copy of \SAGE. 2230 2231 Before using Mercurial, 2232 make sure to define your username and password so the 2233 patches you make are identified as yours. Make a 2234 file \verb+~/.hgrc+ in your home directory like this one: 2235 %skip 2236 \begin{verbatim} 2237 [ui] 2238 username = Cardinal Fang <fang@spanish-inquisition.com> 2239 \end{verbatim} 2240 2241 2242 2243 \section{Quick Mercurial Tutorial for \SAGE} 2244 2245 There are several ways to run Mercurial: from the command line, run 2246 \code{sage -hg} (\code{Hg} is the chemical symbol for mercury), or 2247 from within \sage, run \code{hg_sage}. Most of the examples below use 2248 the second method. 2249 2250 Before you modify \sage library files, you might want to create a copy 2251 of the \sage library in which to work. Do this by typing \code{sage 2252 -clone myver}, for example; then \SAGE will use Mercurial to clone the 2253 current repository and call the result \code{myver}. The new 2254 repository is stored in \code{<SAGE_ROOT>/devel/sage-myver}, and when 2255 you clone, the symlink \verb+sage --> sage-myver+ is made. 2256 2257 (You can also do, e.g., \code{sage -clone -r 1250 oldver}, to get a 2258 clone of \SAGE but as it was at revision 1250. Of course, dependency 2259 issues could make old versions not work (e.g., maybe an old \SAGE 2260 library won't compile with the latest Singular library, which is what 2261 is installed elsewhere in SAGE_ROOT). From with \sage, type 2262 \code{hg_sage.log()} to see the revision history. \note{If you clone 2263 an old version, all of the Cython code is rebuilt, since there is no 2264 easy way to know which files do and don't need rebuilding.}) 2265 2266 Once you've copied the library to a new branch \code{myver} and edited 2267 some files there, you should build the \sage library to incorporate 2268 those changes: type \code{sage -b myver}, or just \code{sage -b} if 2269 the branch \code{myver} is already the current branch: that is, if 2270 \code{<SAGE_ROOT>/devel/sage} links to 2271 \code{<SAGE_ROOT>/devel/sage-myver}. You can also type \code{sage -br 2272 myver} to build the library and then to immediately run \sage. 2273 2274 If you want to submit your changes to the \sage development team for 2275 refereeing (and inclusion into \sage if the referee's report is 2276 positive), you should produce patch files. To do this: 2277 3214 2278 \begin{itemize} 3215 \item {\tt SAGE_ROOT/devel/sage-*} -- the \SAGE library source code 3216 \item {\tt SAGE_ROOT/devel/doc-*} -- the \SAGE documentation 3217 \item {\tt SAGE_ROOT/data/extcode} -- new code included with \SAGE that is written for the systems \SAGE interfaces with, e.g., GAP, PARI, etc. 3218 \item {\tt SAGE_ROOT/local/bin} -- the \SAGE shell scripts 2279 2280 \item Type \code{hg_sage.status()} and \code{hg_sage.diff()} to see 2281 exactly what you've done (you can pass options to \code{diff} to see 2282 information about certain files). 2283 2284 \item If you've added new files, not just edited existing ones, type 2285 \code{hg_sage.add([filenames])} to add those new files to your 2286 repository. 2287 2288 \item Commit your changes by typing \code{hg_sage.commit([optional 2289 filenames])} to commit the changes in files to the repository --- if no 2290 filenames are given, all files are committed. First the output of 2291 \code{hg diff} is displayed: look at it or just enter \code{q}. Then 2292 you are dumped into an editor to type a brief comment on the changes. 2293 The default editor is vi, so type \code{i}, write some meaningful one 2294 line description, hit \code{Escape} and type \code{:wq}. (In bash, to 2295 make emacs the default editor, type \code{export EDITOR=emacs}.) 2296 2297 \item Now create a patch file using \code{hg_sage.export(...)}. This 2298 command needs a revision number (or list of revision numbers) as an 2299 argument; use \code{hg_sage.log()} to see these numbers. An optional 2300 second argument to \code{hg_sage.export(...)} is a filename for the 2301 patch; the default is \code{(changeset_revision_number).patch}. 2302 2303 \item Then post your patch on the \sage trac server: see 2304 Chapter~\ref{ch:trac}. 2305 3219 2306 \end{itemize} 3220 2307 3221 Make sure to define your username and password so the 3222 patches you make are identified as yours. Make a 3223 file \verb+~/.hgrc+ in your home directory like this one: 2308 Note that you can also start a very nice web server that allows you to 2309 navigate your repository with a web browser, or pull patches from it 2310 remotely, by typing \code{hg_sage.serve()}. 2311 2312 Finally, if you want to apply a patch file (perhaps you've downloaded 2313 a patch from the trac server for review), use the command 2314 \code{hg_sage.patch('filename')}. 2315 2316 2317 2318 \section{Using Mercurial with Other \sage Repositories} 2319 2320 \SAGE includes these Mercurial repositories: 2321 \begin{itemize} 2322 \item {\tt SAGE_ROOT/devel/sage-*} : the \SAGE library source code 2323 \item {\tt SAGE_ROOT/devel/doc-*} : the \SAGE documentation 2324 \item {\tt SAGE_ROOT/data/extcode} : external system code, i.e., code 2325 included with \SAGE that is written for the systems with which \SAGE 2326 interfaces, e.g., GAP, PARI, etc. 2327 \item {\tt SAGE_ROOT/local/bin} : the \SAGE shell scripts 2328 \end{itemize} 2329 2330 The previous section discussed using Mercurial with the \sage 2331 library, via the command \code{hg_sage}. There are corresponding 2332 commands for each of the repositories: 2333 2334 \begin{itemize} 2335 \item use \code{hg_sage} for the \sage library 2336 \item use \code{hg_doc} for the \sage documentation 2337 \item use \code{hg_extcode} for the external system code 2338 \item use \code{hg_scripts} for the \sage shell scripts 2339 \end{itemize} 2340 2341 To produce patches for \sage's documentation, for example, use the 2342 same method as outlined above, but with commands like 2343 \code{hg_doc.status()}, \code{hg_doc.commit()}, etc. 2344 2345 2346 2347 2348 \chapter{Producing New \sage Packages} 2349 \label{ch:spkg} 2350 2351 If you are producing code to add new functionality to \sage, you might 2352 consider turning it into a package (an \code{spkg} file) instead of a 2353 patch file. If your code is very large (for instance) and should be 2354 offered as an optional download, a package is the right choice; 2355 similarly, if your code depends on some other optional component of 2356 \sage, you should produce a package. 2357 2358 If you're not sure whether to build an spkg file or a patch file, ask 2359 for advice on \code{sage-devel}. 2360 2361 2362 2363 2364 \section{Creating a New spkg File} 2365 2366 \sage packages are distributed as .spkg files, but an .spkg file is 2367 just a .tar.bz2 file (or a tar file), but named with .spkg to 2368 discourage confusion. In particular, you can type 2369 %skip 3224 2370 \begin{verbatim} 3225 [ui] 3226 username = William Stein <wstein@gmail.com> 2371 tar jxvf mypackage-version.spkg 2372 \end{verbatim} 2373 to extract one and see what is inside. 2374 2375 Here's how to make your own spkg file: 2376 \begin{itemize} 2377 \item[(a)] 2378 Make a directory, e.g., \code{mypackage-0.1}. 2379 The name of the directory should be a lower-case string with no 2380 dashes, followed by a dash, followed by a version number. 2381 2382 \item[(b)] 2383 Put your files in that directory. 2384 2385 \item[(c)] 2386 Create an executable shell script \code{mypackage-0.1/spkg-install}. 2387 2388 \item[(d)] 2389 The script \code{spkg-install} is run during installation 2390 of the \sage package. In this script you may 2391 make the following assumptions: 2392 \begin{itemize} 2393 \item The PATH has the locations of \code{sage} and \code{python} 2394 (from the \sage installation) at the front. Thus the command 2395 %skip 2396 \begin{verbatim} 2397 python setup.py install 2398 \end{verbatim} 2399 will run the correct version of Python with everything set up 2400 correctly. Also, running \code{gap} or \code{Singular}, for example, 2401 will run the correct version. 2402 2403 \item 2404 The environment variable \code{SAGE_ROOT} points to 2405 the root directory of the \sage installation. 2406 \item 2407 The environment variable \code{SAGE_LOCAL} points to 2408 the \code{SAGE_ROOT/local} directory of the \sage installation. 2409 \item 2410 The environment variables 2411 \code{LD_LIBRARY_PATH} and \code{DYLD_LIBRARY_PATH} 2412 both have \code{SAGE_ROOT/local/lib} at the front. 2413 2414 \end{itemize} 2415 2416 \item[(e)] 2417 The \code{spkg-install} script should copy your files to the appropriate 2418 place after doing any build that is necessary. That's it! 2419 2420 \item[(f)] (Optional) Post a copy on the \sage trac server 2421 (Chapter~\ref{ch:trac}) so it can be refereed; if it gets a positive 2422 review, it might be included into the core \sage library, or it might 2423 become an optional download from the \sage web site, so anybody can 2424 automatically install it by typing \code{sage -i mypackage-version.spkg}. 2425 \end{itemize} 2426 2427 If your package depends on another package, say GAP, then you should 2428 check that this other package has been installed. In the case of GAP, 2429 it is installed into the directory \code{SAGE_ROOT/local/lib/gap-4.4.10/} 2430 and your \code{spkg-install} script should check that this exists, 2431 with the code like the following: 2432 %skip 2433 \begin{verbatim} 2434 if [ ! -d $SAGE_ROOT/local/lib/gap-4.4.10 ]; then 2435 echo "This package requires that GAP 4.4.10 is installed." 2436 exit 1 2437 fi 3227 2438 \end{verbatim} 3228 2439 3229 \section{Quick Mercurial tutorial for \SAGE} 3230 3231 Mercurial is available by default since \SAGE version 1.3.7.3. It's confusingly 3232 called {\tt hg} (the chemical symbol for mercury) at the command line. 3233 Do "sage -hg" to run it, or make a link to \verb+SAGE_ROOT/local/bin/hg+ 3234 3235 The file \verb+<SAGE_ROOT>/local/lib/python/site-packages/sage+ 3236 is just a symbolic link to 2440 \emph{Caveat}: Do {\em not} just 2441 copy to \code{SAGE_ROOT/local/lib/gap*/} since that will copy 2442 your package to the lib directory of the {\em old} version of GAP if 2443 GAP is upgraded. 3237 2444 3238 \verb+<SAGE_ROOT>/devel/sage/build/sage+ 2445 External Magma code goes in 2446 \verb+SAGE_ROOT/data/extcode/magma/user+, so if you want to redistribute 2447 Magma code with \sage as a package that Magma-enabled users can use, 2448 that's where you'd put it. You'd also want to have relevant Python code 2449 to make the Magma code easily usable. 3239 2450 3240 Thus by changing what\verb+ <SAGE_ROOT>/devel/sage+ points to, 3241 you can easily switch between running different versions of the 3242 \SAGE library. The "sage -b" command changes this symbolic link 3243 for you automatically. Without argument "sage -b" switches to 3244 where the symbolic link \verb+sage+ points to and builds \SAGE 3245 in this branch. 2451 \textbf{Example}: 2452 These instructions provide directions for creating a particular 2453 \sage package. 3246 2454 3247 If you type \code{sage -clone myver}, say, then \SAGE will use3248 hg to clone the current repository and call the result myver.3249 The new repository is stored in \verb+<SAGE_ROOT>/devel/sage-myver+3250 3251 and when you clone the symlink \verb+sage --> sage-myver+3252 3253 is made. Creating clones of a repository should be fairly fast,3254 e.g., about 30 seconds.3255 You can also do, e.g.,3256 3257 \verb+sage -clone -r 1250 oldver+3258 3259 to get a clone of \SAGE but as it was at revision 1250. Of course,3260 dependency issues could make old versions not work (e.g., maybe an3261 old \SAGE library won't compile with the latest Singular library, which3262 is what is installed elsewhere in SAGE_ROOT).3263 Type \code{hg_sage.log()} to see3264 the revision history.3265 \note{All the cython code is rebuilt3266 if you clone an old version, since there is no easy way to3267 know which files do and don't need rebuilding.}3268 3269 To switch to a different repository and build it, type, e.g.,3270 \verb+sage -b myver+3271 if the repository is stored in3272 3273 \verb+<SAGE_ROOT>/devel/sage-myver+3274 3275 To switch back type \verb+sage -b main+ to the repository stored in3276 \verb+<SAGE_ROOT>/devel/sage-main+3277 3278 3279 You can also use \code{hg} directly from the command line3280 instead of using it via the \code{hg_sage} object.3281 3282 Use \code{sage -hg [options]} from within the repository.3283 See \code{hg(1)} in the man pages for option details.3284 3285 If you want to do a bunch of development, then make3286 it available to others, here is how:3287 2455 \begin{itemize} 3288 2456 \item 3289 Make a new branch (using \code{sage -clone myver}). 2457 Download \code{guava3.6.tar.gz} (the tarball for GUAVA, the 2458 GAP error-correcting codes package) from 2459 \url{http://www.gap-system.org/Packages/guava.html}. 2460 3290 2461 \item 3291 Implement your functionality there. 2462 Suppose you have a directory \code{sagefiles}. 2463 Within that, create a subdirectory \code{guava-3.6}. 2464 Extract the tarball into the directory \code{guava-3.6}. 2465 3292 2466 \item 3293 Type \code{hg_sage.status()} and 3294 \code{hg_sage.diff()} to see exactly what 3295 you've done (you can pass options to diff to 3296 see information about certain files). 2467 Create a file \code{guava-3.6/spkg-install} with the following 2468 contents: 3297 2469 %skip 3298 2470 \begin{verbatim} 3299 -------------------------------------------------------- 3300 | SAGE Version 1.6.1, Build Date: 2007-01-14 | 3301 | Distributed under the GNU General Public License V2. | 3302 -------------------------------------------------------- 2471 #!/bin/sh 3303 2472 3304 Loading SAGE library. Current Mercurial branch is: myver 2473 if [ ! -d $SAGE_ROOT/local/lib/gap-4.4.10 ]; then 2474 echo "This package requires that GAP 4.4.10 is installed." 2475 exit 1 2476 fi 3305 2477 3306 sage: hg_sage.status() 3307 Getting status of modified or unknown files: 3308 cd "/home/jaap/sage/devel/sage" && hg status 3309 M sage/databases/sloane_functions.py 2478 cp -pr guava-3.6 $SAGE_ROOT/local/lib/gap-4.4.10/pkg/ 3310 2479 3311 --- 3312 3313 Branch: myver 3314 3315 sage: hg_sage.diff() 3316 cd "/home/jaap/sage/devel/sage" && hg diff | less 2480 cd $SAGE_ROOT/local/lib/gap-4.4.10/pkg/guava-3.6 2481 ./configure ../.. 2482 make 3317 2483 \end{verbatim} 3318 2484 3319 \item Type \code{hg_sage.add([filenames])} to add new 3320 files to your repository. 3321 \item Type \code{hg_sage.commit([optional filenames])} 3322 to commit the changes in files to the repository--if no filenames are given all 3323 files are committed. First the output of hg diff is piped through less, look at 3324 it or just enter "q". Then you are asked to give your comment on the changes. 3325 Type "i", write some meaningful one liner, hit 'Escape' and type ":wq". 3326 (Note: vi is the default editor. In bash, to make emacs the default, type 3327 \code{export EDITOR=emacs}.) 3328 %skip 3329 \begin{verbatim} 3330 sage: hg_sage.commit() 3331 cd "/home/jaap/sage/devel/sage" && hg diff | less 3332 cd "/home/jaap/sage/devel/sage" && hg commit 3333 \end{verbatim} 2485 Make it executable with \code{chmod +x spkg-install}. 2486 3334 2487 \item 3335 Pull your changes back into your main repository: Note the argument to 3336 hg_sage.pull() can be the full name starting with a '/' or the branch 3337 name 'myver'. 2488 In the directory \code{sagefiles}, 2489 issue the command 3338 2490 3339 2491 %skip 3340 2492 \begin{verbatim} 3341 sage -br main 2493 sage -pkg guava-3.6 2494 \end{verbatim} 2495 This will do some checks, run tar/bz2, and create the spkg file, 2496 \code{guava-3.6.spkg}, in the 2497 directory \code{sagefiles}, 2498 \end{itemize} 3342 2499 3343 sage: hg_sage.pull('myver') #or hg_sage.pull('/home/jaap/sage/devel/sage-myver') 3344 cd "/home/jaap/sage/devel/sage" && hg status 3345 cd "/home/jaap/sage/devel/sage" && hg status 3346 cd "/home/jaap/sage/devel/sage" && hg pull -u /home/jaap/sage/devel/sage-myver 3347 pulling from /home/jaap/sage/devel/sage-myver 3348 searching for changes 3349 adding changesets 3350 adding manifests 3351 adding file changes 3352 added 1 changesets with 1 changes to 1 files 3353 1 files updated, 0 files merged, 0 files removed, 0 files unresolved 3354 If it says use 'hg merge' above, then you should 3355 type hg_sage.merge(), where hg_sage is the name 3356 of the repository you are using. This might not 3357 work with the notebook yet. 2500 To test this, within the \code{sagefiles} directory, type 2501 %skip 2502 \begin{verbatim} 2503 sage -i guava-3.6.spkg 2504 \end{verbatim} 2505 It should install and compile the guava program in sage. 2506 To test its functionality, one could do the following: 3358 2507 3359 sage: hg_sage.update() 3360 cd "/home/jaap/sage/devel/sage" && hg update 3361 0 files updated, 0 files merged, 0 files removed, 0 files unresolved 2508 \begin{verbatim} 2509 sage: gap(`LoadPackage("guava")') # requires optional package 2510 true 2511 sage: gap(`HammingCode(3,3)') # requires optional package 2512 a linear [13,10,3]1 Hamming (3,3) code over GF(3) 3362 2513 \end{verbatim} 3363 2514 3364 \item 3365 You can make {\it all} changes in the repository you're working in 3366 as a bundle by typing \code{hg_sage.bundle('mybundle')} (this 3367 creates an hg bundle \code{mybundle.hg}). Alternatively, 3368 you can export any particular changeset as plain text patches by 3369 typing \code{hg_sage.export(...)}; note that each individual changeset 3370 is recorded as a different patch. 3371 \code{hg_sage.export(...)} needs at least the argument 3372 revs -- integer or list of integers (revision numbers); use the hg_sage.log() 3373 function to see them. An optional second argument is a 'patch_filename', 3374 default is '(changeset_revision_number).patch'. 3375 3376 These can be applied to any other 3377 repository using \code{hg_sage.import_patch('patch_filename')}. 3378 3379 Besides \code{hg_sage.diff(...)}, another way to see what 3380 changes are contained in a patch bundle is to use the command 3381 \code{hg_sage.inspect('patchname.hg')}. This displays the 3382 ``diff file'' as plain text using \code{less}. 3383 3384 Note that you don't get to ``cherry pick'' what you send in a bundle. 3385 Everything you recorded is sent. Hence you might want to use the 3386 cloning, etc. above to send only from your own sage-main repository. 3387 Note also that if you have a repository, e.g., sage-mynewstuff 3388 and it's all ready to use, you can do 3389 \verb+sage -hg pull <SAGE_ROOT>/devel/sage-mynewstuff+ 3390 from your \verb+<SAGE_ROOT>/devel/sage-main repository+, and you'll 3391 get all the sage-mynewstuff merged in. NOTE: read the output! 3392 After that you must type \code{sage -hg update}. 3393 3394 \item 3395 You can also start a very nice web server that allows you to 3396 navigate your repository with a web browser, or pull patches from it 3397 remotely: 3398 2515 To build it again if you've already installed the package, use 3399 2516 %skip 3400 2517 \begin{verbatim} 3401 sage: hg_sage.serve() 2518 sage -f guava-3.6.spkg 3402 2519 \end{verbatim} 3403 2520 3404 \item3405 For the occasional contributor:3406 %skip3407 \begin{verbatim}3408 sage: hg_sage.ci()3409 sage: hg_sage.send('patchname.hg')3410 \end{verbatim}3411 2521 3412 Post the file 'patchname.hg' to the trac server. 3413 Note that 'ci()' is an alias for 'commit()'. 2522 2523 2524 \chapter{The \sage Trac Server: Submitting Patches and Packages} 2525 \label{ch:trac} 2526 2527 What should you do with your Mercurial patches for \sage? You should 2528 post them on the \sage trac server. 2529 2530 The \sage trac server, located at 2531 \url{http://trac.sagemath.org/sage_trac/}, is where \sage bugs are 2532 listed and patched, new code is posted and reviewed, and ideas for 2533 extending and improving \sage are discussed. Thus if you find a bug 2534 in \sage, or if you have new code to submit, or if you have 2535 corrections for the documentation, you should post on the trac server. 2536 2537 Items on the server are called ``tickets'', and anyone may browse the 2538 tickets: just visit \url{http://trac.sagemath.org/sage_trac/report}. 2539 You need to open an account, though, if you want to comment on a 2540 ticket, submit a patch, or create a new ticket. See the web page 2541 \url{http://wiki.sagemath.org/TracGuidelines} for more information 2542 about obtaining an account, as well as general guidelines for using 2543 the trac server. 2544 2545 2546 2547 \section{Reporting Bugs} 2548 2549 ``The first step is admitting you have a problem.'' 2550 2551 If you think you've found a bug in \sage, you should first search 2552 through the Google groups \code{sage-devel} and \code{sage-support} 2553 for postings related to your possible bug: maybe it has already been 2554 discussed. You should also search the trac server to see if anyone 2555 else has opened a ticket about your bug. 2556 2557 If you don't find anything, and you're not sure that you've found a 2558 bug, ask about it on \code{sage-devel}. 2559 2560 If you don't find anything, and if you're positive you've found a bug, 2561 open a new ticket on the trac server. As mentioned above, you need an 2562 account to do this. To report a bug: log in and click on the ``New 2563 ticket'' button. Type a meaningful one-liner in the ``Short summary'' 2564 box, with more information in the larger box below. You should 2565 include an explicit, reproducible example illustrating your bug 2566 (and/or the steps required to reproduce the buggy behavior). You 2567 should also include the version of \sage (and any relevant packages) 2568 you are using, and operating system information, being precise as 2569 possible (32-bit, 64-bit, ...). 2570 2571 Between the ``Summary'' and ``Full description'' boxes, there is a 2572 place to choose the ``Type'' of the ticket: ``Defect'', 2573 ``Enhancement'', or ``Task''. Use your best judgment here; a bug 2574 should probably be reported as a ``Defect''. 2575 2576 Choose a priority for your bug, keeping in mind that the ``blocker'' 2577 label should be used very sparingly. Also pick a component for your 2578 bug; this is sometimes straightforward -- if your bug deals with 2579 \sage's calculus implementation, choose ``calculus''. If it is not 2580 obvious, do your best. Choose a milestone; if you're not sure what to 2581 choose, just choose the numbered version of sage from the menu 2582 (``sage-3.1.4'', for example). Type in some helpful keywords. In the 2583 box labeled ``Assign to'', type ``somebody'' if you're not sure what 2584 else to do. 2585 2586 Hit the ``Preview'' button to make sure everything looks okay, and 2587 then hit ``Submit ticket''. 2588 2589 2590 2591 \section{Guidelines for Opening Tickets} 2592 2593 In addition to bug reports, you should also open a ticket if you have 2594 some new code which extends \sage's capabilities. If you have a 2595 feature request, start a discussion on \code{sage-devel} first, and 2596 then if there seems to be general agreement that you have a good idea, 2597 open a ticket describing the idea. 2598 2599 Other comments: 2600 \begin{itemize} 2601 \item Before opening a ticket, make sure that nobody else has opened a 2602 ticket about the same or closely related issue. 2603 \item It is much better to open several specific tickets than one that 2604 is very broad. Indeed, a single ticket which deals with lots of 2605 different issues can be quite problematic, and should be avoided. 2606 \item Be precise: If foo doesn't work on OS X but is fine on Linux, 2607 mention that in the title; also use the keyword option so that 2608 searches will pick up the issue. 2609 \item The problem described in the ticket must be solvable. For 2610 example, it would be silly to open a ticket whose purpose was ``Make 2611 \sage the best mathematical software in the world''. There is no metric 2612 to measure this properly and it is highly subjective. 3414 2613 \end{itemize} 3415 You can browse the official repositories for \SAGE with your web browser here:3416 \verb+http://sagemath.org/sage/hg+3417 These are served using a cgi-bin script.3418 3419 2614 3420 \section{Updating To the Latest Official \SAGE Library Source Code}3421 2615 3422 If you're wary of messing things up, before typing the commands3423 explained below, first type from the command line3424 \code{sage -clone test}. To switch back to your main repository (the default),3425 type \code{sage -b main}.3426 2616 3427 To update to the latest \SAGE Mercurial source 3428 code type \code{hg_sage.pull()}. You may have to type 3429 \code{hg_sage.update()}. 2617 \section{Patching Bugs/Working on Tickets} 3430 2618 3431 At this stage, you might get a message like: 3432 \begin{verbatim} 3433 remote changed .hgtags which local deleted 3434 (k)eep or (d)elete? 3435 \end{verbatim} 3436 Type \verb+k <RETURN>+. You might then get 3437 \begin{verbatim} 3438 remote changed export which local deleted 3439 (k)eep or (d)elete? 3440 \end{verbatim} 3441 Type \verb+k <RETURN>+. 2619 If you have code which fixes a bug or deals with some issue in \sage, 2620 here's what to do: first, use Mercurial to create a patch file. If 2621 the issue has been reported as a ticket on the trac server, attach your 2622 patch file to that ticket: go to the ticket, click on the ``Attach 2623 File'' button, and follow the directions. On the ticket page, you 2624 should add a comment explaining your patch, and you should also 2625 change the summary for the ticket from \texttt{description of bug} to 2626 \texttt{[with patch, needs review] description of bug}. 3442 2627 3443 You might then get 3444 \verb+merging sage/...+ 3445 Now kdiff3 opens with some text highlighted 3446 in yellow and red. The bar at the bottom says "Number of remaining 3447 unsolved conflicts: ...". 3448 The conflict appears to be in the file named in the lower part 3449 of the kdiff3 window. Clicked on "merge" on the 3450 top bar, pick one of the options in the pop-down menu, 3451 then click on "save" and quit. 2628 If there is no trac ticket associated to this issue, create one (as 2629 explained in the previous sections) describing the issue and your 2630 solution, attach your patch, and give it a summary of the form 2631 \texttt{[with patch, needs review] description of bug goes here}. 3452 2632 3453 If you don't have kdiff3, install it with \code{apt get kdiff3} or3454 \code{yum install kdiff3}.3455 2633 3456 If it says you need to, type3457 \code{hg_sage.commit()}3458 to finish updating your mercurial repository.3459 2634 2635 \section{Reviewing Patches} 3460 2636 3461 \chapter{Miscellaneous} 3462 \section{Circular Imports} 3463 This needs to be cleaned up, but an irc log about this common problem 3464 is better than nothing: 3465 \begin{verbatim} 3466 ncalexan1: from sage.schemes.plane_curves.curve import Curve_generic 3467 [5:08pm] ncalexan1: gives an ImportError 3468 was389: You can import anything from a file *unless* you set yourself up to 3469 have a *circular* import loop. 3470 [5:11pm] was389: The interpreter actually runs through the code of the file, 3471 so it's not possible to circularly import usig 3472 [5:11pm] ncalexan1: That's possible. 3473 [5:11pm] was389: from blah import foo 3474 [5:11pm] ncalexan1: Oh. 3475 [5:11pm] was389: You might have to either (1) do import 3476 sage.schemes.plane_curves.curve and explicitly access 3477 [5:12pm] was389: Curve_generic, or (2) wait to import Curve_generic until 3478 where it is used. 3479 \end{verbatim} 2637 All code that goes into \sage is peer-reviewed, to ensure that the 2638 conventions discussed in this manual are followed, to make sure that 2639 there are sufficient examples and doctests in the documentation, and 2640 to try to make sure that the code does, mathematically, what it is 2641 supposed to. 3480 2642 3481 \section{Weird Issues} 3482 \begin{enumerate} 3483 \item 3484 If you see3485 \begin{verbatim} 3486 > > - "*** glibc detected *** free(): invalid pointer: 0x0846a684 ***" 3487 \end{verbatim} 3488 when leaving \sage, you might wonder why.2643 If someone (other than you) has posted a patch for a ticket on the 2644 trac server, you can review it. Look at the patch (by clicking on the 2645 file name in the list of attachments) to see if it makes sense. 2646 Download it (from the window displaying the patch, see the 2647 ``Download'' option at the bottom of the page). Apply it (using 2648 \code{hg_sage.patch('filename')}, for example) to your copy of \sage, 2649 and build \sage with the new code by typing \code{sage -b} (for 2650 patches to .py files) or \code{sage -ba} (for patches to .pyx files). 3489 2651 3490 Martin Albrecht finally figured out how to solve this strange bug. The 3491 scenario: If you write a PYREX wrapper for some C code which needs to 3492 be linked to the GMP library and you load the extension you wrote 3493 during runtime into \sage you'll get such error messages on exit or 3494 even more strange behavior. 3495 3496 This vanishes completely if you load your extension module during the 3497 startup of \SAGE like the other library wrappers/PYREX code (e.g. in 3498 \code{sage/libs/all.py}). This is because when you load the module the 3499 GMP library gets somehow reinitialized and all your prior GMP 3500 variables are not valid anymore. So when they are freed/cleared on 3501 exit you get this error stated above. Some objects relying on GMP are 3502 created during startup of \SAGE so you'll see this behavior as well if 3503 the first thing you do on the prompt is to load your library. 2652 Now ask yourself questions like these: 2653 \begin{itemize} 2654 \item Does the new source code make sense? 2655 \item When you run it in \sage, does it fix the problem reported on 2656 the ticket? 2657 \item Does it fail to introduce any new problems? 2658 \item Is it documented sufficiently, including both explanations and 2659 doctests? (This is \textbf{very} important: all code in \sage must 2660 have doctests, so even if the patch is for code which didn't have a 2661 doctest before, the new version must include one.) 2662 \item In particular, is there a doctest illustrating that the bug has 2663 been fixed? If a function used to give the wrong answer and this 2664 patch fixes that, then if possible, it should include a doctest 2665 illustrating its new success. 2666 \end{itemize} 3504 2667 3505 \end{enumerate} 2668 If the answers to these and other such reasonable questions are yes, 2669 then you might want to give the patch a positive review. On the main 2670 ticket page, write a comment in the box and change the summary from 2671 \texttt{[with patch, needs review] description of bug} to 2672 \texttt{[with patch, positive review] description of bug}. If you 2673 feel there are issues with the patch, explain them in the comment box, 2674 and change the summary to \texttt{[with patch, negative review] 2675 description of bug}, or \texttt{[with patch, needs work] 2676 description of bug}, or \texttt{[with patch, positive review pending fixes] 2677 description of bug}, or something similar. Browse the tickets on the 2678 trac server to see how things are done. 3506 2679 3507 \section{Benchmarking} 3508 \label{sec:Benchmarking} 2680 By the way, if you review a patch which deals with the \sage manuals, 2681 say, instead of the source code, then you need to use 2682 \code{hg_doc.patch('filename')} instead of 2683 \code{hg_sage.patch('filename')} to apply it, and you need to follow 2684 the directions in Chapter~\ref{ch:manuals} to build the new 2685 documentation. 3509 2686 3510 \begin{verbatim}3511 2687 3512 http://sage.math.washington.edu:8100/william_32bit_add?edit 2688 \section{Closing Tickets} 3513 2689 3514 {{{ 3515 if is_64_bit: 3516 word = 64 3517 else: 3518 word = 32 3519 }}} 2690 Don't close tickets. That is the job of the \sage administrators. If 2691 you feel strongly that a ticket should be closed or deleted, send 2692 email to the current release manager explaining the situation. 3520 2693 3521 {{{3522 def testpy(n, op):3523 a = long(ZZ.random(2^(word*n)))3524 b = long(ZZ.random(2^(word*n)))3525 N = 10000003526 t = cputime()3527 for i in range(N):3528 c = op(a, b)3529 return cputime(t)3530 }}}3531 2694 3532 {{{3533 def testgmp(n, op):3534 a = ZZ.random(2^(word*n))3535 b = ZZ.random(2^(word*n))3536 N = 10000003537 t = cputime()3538 for i in range(N):3539 c = op(a, b)3540 return cputime(t)3541 }}}3542 2695 3543 {{{3544 op=operator.add3545 for n in range(1,20):3546 print 4*n, testpy(4*n,op) / testgmp(4*n,op)3547 ///3548 4 0.3366336633663549 8 0.3786407766993550 12 0.3883495145633551 16 0.4509803921573552 20 0.5098039215693553 24 0.5092592592593554 28 0.5272727272733555 32 0.5045871559633556 36 0.6538461538463557 40 0.5904761904763558 44 0.6818181818183559 48 0.7181818181823560 52 0.6548672566373561 56 0.6486486486493562 60 0.9734513274343563 64 0.9568965517243564 68 0.9203539823013565 72 1.078947368423566 76 1.043478260873567 }}}3568 2696 3569 {{{3570 op=operator.mul3571 for n in range(1,10):3572 print n, testpy(n,op) / testgmp(n,op)3573 ///3574 1 0.4753575 2 0.53576 3 0.5277777777783577 4 0.6637168141593578 5 0.8017241379313579 6 0.9915966386553580 7 1.215517241383581 8 1.411764705883582 9 1.565891472873583 }}}3584 2697 3585 \ end{verbatim}2698 \appendix 3586 2699 3587 \begin{verbatim} 3588 3589 http://sage.math.washington.edu:8100/william_64bit_add 3590 3591 3592 {{{ 3593 if is_64_bit: 3594 word = 64 3595 else: 3596 word = 32 3597 print word 3598 /// 3599 64 3600 }}} 3601 3602 {{{ 3603 def testpy(n, op): 3604 a = long(ZZ.random(2^(word*n))) 3605 b = long(ZZ.random(2^(word*n))) 3606 N = 1000000 3607 t = cputime() 3608 for i in range(N): 3609 c = op(a, b) 3610 return cputime(t) 3611 }}} 3612 3613 {{{ 3614 def testgmp(n, op): 3615 a = ZZ.random(2^(word*n)) 3616 b = ZZ.random(2^(word*n)) 3617 N = 1000000 3618 t = cputime() 3619 for i in range(N): 3620 c = op(a, b) 3621 return cputime(t) 3622 }}} 3623 3624 {{{ 3625 op=operator.add 3626 for n in range(1,20): 3627 print 4*n, testpy(4*n,op) / testgmp(4*n,op) 3628 /// 3629 4 0.478991596639 3630 8 0.5 3631 12 0.526717557252 3632 16 0.55905511811 3633 20 0.5859375 3634 24 0.615384615385 3635 28 0.709923664122 3636 32 0.731884057971 3637 36 0.833333333333 3638 40 0.873134328358 3639 44 0.878571428571 3640 48 0.85401459854 3641 52 0.950704225352 3642 56 1.1 3643 60 1.12121212121 3644 64 0.977941176471 3645 68 3646 }}} 3647 3648 {{{ 3649 op=operator.mul 3650 for n in range(1,10): 3651 print n, testpy(n,op) / testgmp(n,op) 3652 /// 3653 1 0.487179487179 3654 2 0.747826086957 3655 3 1.13913043478 3656 4 1.56896551724 3657 5 2.0859375 3658 6 2.63846153846 3659 7 3.35384615385 3660 8 4.38636363636 3661 9 5.0962962963 3662 }}} 3663 3664 3665 \end{verbatim} 3666 3667 \chapter{History and License} 2700 \chapter{\sage's License} 3668 2701 3669 2702 \input{license} 3670 2703