Opened 10 years ago

Last modified 5 years ago

#8290 needs_info enhancement

Support HDF5

Reported by: magawake Owned by: GeorgSWeber
Priority: major Milestone: sage-6.4
Component: packages: optional Keywords: hdf5 h5py hdf5python
Cc: Merged in:
Authors: Andrzej Giniewicz Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by aginiewicz)

We would like to have hdf5 support into sage. This will include the C/C++ bindings and also should have the Java bindings.

To obtain HDF5: http://www.hdfgroup.org/HDF5/release/obtain5.html

Current version of spkg (install in order):

Change History (27)

comment:1 Changed 10 years ago by was

Here is a quick spkg that works if you have libhdf5 devel packages installed on your computer:

http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

Install it with

sage -i http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

comment:2 Changed 10 years ago by magawake

Thanks. Couple of requirements

  • Have it completely independent therefore be shipped with SAGE
  • Have all the C/C++ bindings includes into the SAGE tar ball

comment:3 Changed 10 years ago by magawake

  • Keywords hdf5python added

comment:4 Changed 10 years ago by was

I also made an hdf5 library spkg, but the hdf5 version is evidently too new for h5py:

http://www.hdfgroup.org/ftp/HDF5/prev-releases/hdf5-1.8.0/src/hdf5-1.8.0.tar.gz

since compiling this, then h5py does not work.

comment:5 Changed 10 years ago by was

  • Status changed from new to needs_review

Hi,

So if you build

http://sage.math.washington.edu/home/wstein/patches/hdf5-1.6.9.spkg

and

http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

then it should work. Example, do:

   sage -f  http://sage.math.washington.edu/home/wstein/patches/hdf5-1.6.9.spkg  http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

I'm posting this for inclusion in the *experimental* repo.

comment:6 Changed 10 years ago by was

Example:

Here's a trivial example showing how to create a new HDF5 file and store a 100 x 20 array of floats:

>>> f = h5py.File("myfile.hdf5", 'w')
>>> f["MyDataset"] = numpy.ones((100,20))

And to get your data back:

>>> dset = f["MyDataset"]
>>> subset = dset[20:80,:]

See http://code.google.com/p/h5py/

comment:7 Changed 10 years ago by magawake

Is it possible to have atleast hdf 1.8.2? 1.6.x is too old. Most of our files are 1.8.x format.

comment:9 Changed 10 years ago by jason

Also, on the front page of the h5py project, it says "Transparently supports both HDF5 1.6 and 1.8.".

comment:10 Changed 10 years ago by jason

On http://code.google.com/p/h5py/wiki/FAQ, it says that Linux supports up to 1.8.3.

comment:11 Changed 10 years ago by jason

On http://code.google.com/p/h5py/wiki/FAQ, it says that Linux supports up to 1.8.3.

comment:12 Changed 9 years ago by aginiewicz

actually h5py should support hdf5 up to 1.8.4 an PyTables? up to 1.8.3. I'd say sage should use hdf5 1.8, it's first version to introduce external links and utf8 encoded strings so two things that seems to be quite important, right?

comment:14 Changed 9 years ago by aginiewicz

I made quick update of those to hdf5 1.8, also this version of h5py works with hdf5 from spkg (at least it works for me - only had to specify "--hdf=$SAGE_LOCAL" to build script).

http://lab15.im.pwr.wroc.pl/~giniew/h5py-1.3.0.spkg

http://lab15.im.pwr.wroc.pl/~giniew/hdf5-1.8.4.spkg (actually, it's 1.8.4 patch 1, latest supported by h5py at time of writing - wasn't sure if I should add the patch1 to name here).

They are based on packages posted here earlier - just updated the src to pointed h5py to local hdf5 copy.

comment:15 Changed 8 years ago by mhansen

  • Status changed from needs_review to needs_info

Ping. These spkgs no longer exist.

comment:16 Changed 8 years ago by aginiewicz

  • Status changed from needs_info to needs_review

Well, after a year I forgot about it - the files are present, but url changed. For now I don't have time to update them though (h5py is now 2.0.1 and hdf5 is 1.8.8). Anyway, working links:

http://im.pwr.wroc.pl/~giniew/h5py-1.3.0.spkg

and

http://im.pwr.wroc.pl/~giniew/hdf5-1.8.4.spkg

comment:17 Changed 7 years ago by aginiewicz

  • Authors set to aginiewicz
  • Description modified (diff)

I've updated spkgs with latest versions. Also, those spkgs now contains repositories and better descriptions in SPKG.txt files (description, license, dependencies, changelog - usual stuff). I've also added links to description to not hunt for them in comments

comment:18 Changed 7 years ago by jdemeyer

Please fill in your real name as Author.

comment:19 Changed 7 years ago by aginiewicz

  • Authors changed from aginiewicz to Andrzej Giniewicz

comment:20 Changed 7 years ago by jdemeyer

  • Component changed from build to packages
  • Status changed from needs_review to needs_info

Are these supposed to be standard packages, optional packages or experimental packages?

comment:21 Changed 7 years ago by aginiewicz

In its current state (i.e. only hdf5+h5py) it might be not that useful to include in standard spkg. I believe it could be optional package. But if some other packages would be compiled with hdf5 support (especially R, which is standard package) situation might change. This is of course only my opinion and it might be not mirror others opinion.

Anyway, I got e-mail that h5py package got broken during upload to github. I had to recreate and reupload it (now, I will not trust github any more and keep other copy around) - it should work now.

comment:22 Changed 7 years ago by mister.wardrop

This would be really useful for making data interchangeable with other computing platforms. I look forward to seeing this included in Sage by default :).

comment:23 Changed 6 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:24 Changed 6 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:25 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:26 Changed 5 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:27 Changed 5 years ago by jdemeyer

  • Component changed from packages: standard to packages: optional
Note: See TracTickets for help on using tickets.