Changes between Initial Version and Version 1 of Ticket #25262, comment 41


Ignore:
Timestamp:
08/17/18 16:09:34 (4 years ago)
Author:
embray
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #25262, comment 41

    initial v1  
    11I've been poking around a bit more in the ASV source, and am going to see what I can come up with.
    22
    3 1. There is a base `Benchmark` class that I believe we can customize just a little bit for our purposes, and the rest of the code is flexible enough that we should be able to add our custom Benchmark class to the list of known benchmark types (`asv.benchmark.benchmark_types`).  Ideally a plugin would be able to do this without modifying any internal data structures.
     31. There is a base `Benchmark` class that I believe we can customize just a little bit for our purposes, and the rest of the code is flexible enough that we should be able to add our custom Benchmark class to the list of known benchmark types (`asv.benchmark.benchmark_types`).  Ideally a plugin would be able to do this without modifying any internal data structures.  (In fact, now I'm thinking we may not even need any Benchmark subclasses if our custom discovery code is clever enough...)
    44
    552. The main thing, then, that we need to customize is benchmark discovery.  Currently there's no great way to do this and this is where another plugin interface is needed.  The current plugin discovery process ultimately yields `Benchmark` instances which contain all the information needed for a single benchmark test (it also wraps the benchmark function itself--in this case we can either generate a function from the doctest, or use a standard function for running a single doctest).  What we need then is a way to extend the benchmark discovery process to allow discovery from arbitrary sources (rather than just searching the file system for .py files and importing them).