Generic_Util.numba package#

Submodules#

Generic_Util.numba.benchmarking module#

Functions comparing execution times of (semi-automatically-generated) varieties of numba-compilations of a given function, including lazy vs eager compilation, vectorisation, parallelisation, as well as varieties of rolling (see Generic_Util.numba.higher_order).

Generic_Util.numba.benchmarking.compare_compilations(f: Callable | None = None, numba_signature=None, separate_numba_signatures=None, f_scalar: Callable | None = None, numba_signatures_scalar=None, separate_numba_signaturess_scalar=None, parallel=False, fs_with_own_args: dict[str, tuple[Callable, list, dict]] | None = None, n=200, wait=1, prefix=None, verbose=True, args: list | None = None, kwargs: dict | None = None)#

Combine available ingredients to benchmark possible numba versions of the given function.

numba_signature and parallel are only used if f is given. numba_signatures_scalar is only used if f_scalar is given. separate_numba_signatures and separate_numba_signaturess_scalar are, respectively, a list and a list of lists of additional signatures to be tried in addition to, respectively, numba_signature and numba_signatures_scalar.

Generic_Util.numba.benchmarking.compare_rolls(f_pairs: dict[str, tuple[typing.Callable, typing.Callable]], test_array, test_n=200, n=3, nb_type=float64, nb_out_type=None, np_out_type=<class 'numpy.float64'>, input_is_2d=False)#

For each labelled (1-char keys recommended for neatness) implementation of a given function (and possibly a stencil-optimised version of it, otherwise a None in its tuple), first check that all outputs agree and then benchmark the possible stencil variations:

no stencil (NONE), automatic stencil (AUTO) and provided stencil (GIVE)

Generic_Util.numba.higher_order module#

Higher-order numba-compilation functions, currently only functions to “roll” simpler functions (1d-to-scalar or 2d-to-scalar/1d) over arrays, with a few combinations of input and output type signatures.

Generic_Util.numba.higher_order.roll(f, f_stencil_version=None, n=3, nb_type=float64, nb_out_type=None, np_out_type=<class 'numpy.float64'>, use_stencil=True, input_is_2d=False)#

Produce a compiled function which roll-applies a 1d-to-scalar f (and possibly a stencil-able version of f too) to a 1d or 2d input array for the given window size, numba input type (output type too if different) and numpy output type. I.e. a function which applies f to n-slices of the input array (full-width, sliced-height in the 2d case) and returns an array of the same shape as the input. Arguments to the produced function are expected to be F-contiguous if input_is_2d and C-contiguous otherwise (can impose either with np.asfortranarray/np.ascontiguousarray and inspect it with typeof(ARRAY).layout == ‘F/C’), while outputs are always C-contiguous.

NOTE: Unless willing to manually check the first n-1 entries for correctness when applying the produced function, avoid using explicit references to the slice elements by index (e.g. xs[-2] + xs[-1] + xs[0]) in f’s definition; use collective operators instead (e.g. xs[-2:1].sum(), which is fine because slicing deals with missing entries).

Parameters:

  • f – A 1d-to-scalar function compilable as nb_out_type(nb_type[::1]) (or, same but nominally different, nb_out_type(Array(nb_type, 1, ‘F’)) if input_is_2d)

  • f_stencil_version – Optional - A function identical to f except in that it indexes its input with stencil-compatible window indices coherent with the given n, i.e. same code as f but having xs[1-n:1] instead of xs (note the n-1 nature of modifications). If the underlying code is optimised in particular in numba (e.g. numpy vectorised operators and methods), then the output will be considerably faster with this argument than not; HOWEVER, if the code is NOT as optimised in numba, then the output could be considerably SLOWER than without this argument.

  • n – The window size (how many previous xs, including the current one, should be included in the processed slices)

  • nb_type – The scalar numba type of f’s (and the produced function’s) input

  • nb_out_type – Optional - The scalar numba type of f’s (and the produced function’s) output; provide only if different from nb_type

  • np_out_type – The scalar numpy type of f’s (and the produced function’s) output (required for array allocation, while the input one is not)

  • use_stencil – [True by default] Whether the produced function should be built from stencil (and guvectorize) rather than njit; see the caveat in f_stencil_version’s description; the best practice is to provide a f_stencil_version and benchmark the output over use_stencil

  • input_is_2d – Whether the input to the produced function is going to be an F-contiguous 2d array (on which f will be applied column-wise) instead of a C-contiguous 1d array

Returns:

A function applying f to every n-slice (or [n,:]-slice) of the input array, with signature nb_out_type[::1](nb_type[::1]) or nb_out_type[:, ::1](nb_type[::1, :]) if input_is_2d

Generic_Util.numba.higher_order.roll2d(f, n=3, nb_type=float64, nb_out_type=None, np_out_type=<class 'numpy.float64'>, output_is_2d=False)#

Produce a compiled function which roll-applies a 2d-to-scalar or 2d-to-1d(*) f to a 2d input array for the given window size, numba input type (output type too if different) and numpy output type. I.e. a function which applies f to [n,:]-slices of the input array and returns either a 1d or 2d array. Arguments to the produced function are expected to be C-contiguous (can impose it with np.ascontiguousarray and inspect it with typeof(ARRAY).layout == ‘C’), and outputs are always C-contiguous.

(*): The 2d-to-1d function should actually be (2d,1d)-to-void, i.e. it needs to take in the 1d output array as an argument and modify it in-place, returning nothing (providing pre-allocated output arrays this way drastically speeds up the process).

NOTE: Unless willing to manually check the first n-1 entries for correctness when applying the produced function, avoid using explicit references to the slice elements by index (e.g. xs[-2,:] + xs[-1,:] + xs[0,:]) in f’s definition; use collective operators instead (e.g. xs[-2:1,:].sum(), which is fine because slicing deals with missing entries).

Parameters:

  • f – A 2d-to-scalar or (2d,1d)-to-void function, respectively compilable as nb_out_type(nb_type[:, ::1]) or void(nb_type[:, ::1], nb_out_type[::1]) if f_is_1d_void

  • n – The window size (how many previous xs rows, including the current one, should be included in the processed slices)

  • nb_type – The scalar numba type of f’s (and the produced function’s) input

  • nb_out_type – Optional - The scalar numba type of f’s (and the produced function’s) output; provide only if different from nb_type

  • np_out_type – The scalar numpy type of f’s (and the produced function’s) output (required for array allocation, while the input one is not)

  • output_is_2d – Whether, rather than scalar, f’s output is void but it takes a 1d pre-allocated array which it fills with the true output (see type signatures in f description), leading to the produced function’s output being 2d

Returns:

A function applying f to every [n,:]-slice of the input array, with signature nb_out_type[::1](nb_type[:, ::1]) or nb_out_type[:, ::1](nb_type[:, ::1]) if f_is_1d_void

Generic_Util.numba.types module#

Convenient shorthands for frequently used numba (and respective numpy) types, with a focus on C-contiguity of arrays; these are useful in declaring eager-compilation function signatures.

Generic_Util.numba.types.a1dF(nb_type)#
Generic_Util.numba.types.aList#

alias of List

Generic_Util.numba.types.b1A2_NP#

alias of ndarray[Any, dtype[bool_]]

Generic_Util.numba.types.b1A_NP#

alias of ndarray[Any, dtype[bool_]]

Generic_Util.numba.types.b1_NP#

alias of bool_

Generic_Util.numba.types.cA2_NP#

alias of ndarray[Any, dtype[<U1]]

Generic_Util.numba.types.chA_NP#

alias of ndarray[Any, dtype[<U1]]

Generic_Util.numba.types.f8A2_NP#

alias of ndarray[Any, dtype[float64]]

Generic_Util.numba.types.f8A_NP#

alias of ndarray[Any, dtype[float64]]

Generic_Util.numba.types.f8_NP#

alias of float64

Generic_Util.numba.types.guvectorize(ftylist, signature, target='cpu', identity=None, **kws)#

A decorator to create NumPy generalized-ufunc object from Numba compiled code.

Parameters:

  • ftylist (iterable) – An iterable of type signatures, which are either function type object or a string describing the function type.

  • signature (str) – A NumPy generalized-ufunc signature. e.g. “(m, n), (n, p)->(m, p)”

  • identity (int, str, or None) – The identity (or unit) value for the element-wise function being implemented. Allowed values are None (the default), 0, 1, and “reorderable”.

  • cache (bool) – Turns on caching.

  • writable_args (tuple) – a tuple of indices of input variables that are writable.

  • target (str) – A string for code generation target. Defaults to “cpu”.

Return type:

A NumPy generalized universal-function

Example

@guvectorize([‘void(int32[:,:], int32[:,:], int32[:,:])’,

‘void(float32[:,:], float32[:,:], float32[:,:])’], ‘(x, y),(x, y)->(x, y)’)

def add_2d_array(a, b, c):
for i in range(c.shape[0]):
for j in range(c.shape[1]):

c[i, j] = a[i, j] + b[i, j]

Generic_Util.numba.types.i1A2_NP#

alias of ndarray[Any, dtype[int8]]

Generic_Util.numba.types.i1A_NP#

alias of ndarray[Any, dtype[int8]]

Generic_Util.numba.types.i1_NP#

alias of int8

Generic_Util.numba.types.i8A2_NP#

alias of ndarray[Any, dtype[int64]]

Generic_Util.numba.types.i8A_NP#

alias of ndarray[Any, dtype[int64]]

Generic_Util.numba.types.i8_NP#

alias of int64

Generic_Util.numba.types.nList(dtype, reflected=False, initial_value=None)#
Generic_Util.numba.types.nTup(*args)#
Generic_Util.numba.types.njit(*args, **kws)#

Equivalent to jit(nopython=True)

See documentation for jit function/decorator for full description.

Generic_Util.numba.types.stencil(func_or_mode='constant', **options)#
Generic_Util.numba.types.typeof(val, purpose=Purpose.argument)#

Get the Numba type of a Python value for the given purpose.

Generic_Util.numba.types.vectorize(ftylist_or_function=(), target='cpu', identity=None, **kws)#

A decorator that creates a NumPy ufunc object using Numba compiled code. When no arguments or only keyword arguments are given, vectorize will return a Numba dynamic ufunc (DUFunc) object, where compilation/specialization may occur at call-time.

Parameters:

ftylist_or_function (function or iterable) –

When the first argument is a function, signatures are dealt with at call-time.

When the first argument is an iterable of type signatures, which are either function type object or a string describing the function type, signatures are finalized at decoration time.

Keyword Arguments:

  • target (str) – A string for code generation target. Default to “cpu”.

  • identity (int, str, or None) – The identity (or unit) value for the element-wise function being implemented. Allowed values are None (the default), 0, 1, and “reorderable”.

  • cache (bool) – Turns on caching.

Return type:

A NumPy universal function

Examples

@vectorize([‘float32(float32, float32)’,

‘float64(float64, float64)’], identity=0)

def sum(a, b):

return a + b

@vectorize def sum(a, b):

return a + b

@vectorize(identity=1) def mul(a, b):

return a * b

Module contents#