U idף@sdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl mZmZmZmZddlmZmZmZmZmZmZddlZddlmZddlmZedZ z&ddl!m"Z"e"dd l#m$Z$d Z%Wne&k rd Z%YnXzdd l#m'Z'd Z(Wne&k r*d Z(YnXzddl)Z*d Z+Wne&k rVd Z+YnXe ,e(d Z-e ,e%dZ.e(o~e'j/dkZ0e ,e0dZ1ej23dZ4ej23dZ5ej6dkZ7e7Z8e4 Z9ddZ:e+rddZ;nddZ;ddZZ?Gdd d e?Z@Gd!d"d"e>ZAGd#d$d$e?ZBGd%d&d&e?ZCGd'd(d(e?ZDd)d*ZEd+d,ZFe jGZHGd-d.d.e>ZId/d0ZJeFeHejKZLeFeJd1ZMe4seFeJd2ZNeFeJd3ZOeFeJd4ZPGd5d6d6eZQejRd7kZSe ,eSd8Gd9d:d:eQZTeTUGd;d<dZVGd=d>d>eVeQZWeWUGd?d@d@eZXeGdAdBdBeXZYeYUeGdCdDdDeXZZeGdEdFdFeXZ[ee1GdGdHdHeXZ\ee.GdIdJdJeXZ]eGdKdLdLeZ^ee-GdMdNdNeZ_e`dOkre adS)Pz Tests the parallel backend N)jit vectorize guvectorizeset_num_threads)temp_directoryoverride_configTestCasetagskip_parfors_unsupported linux_only)_TIMEOUT)configgN@)_check_tbb_version_compatible)tbbpoolTF)omppoolzOpenMP threadpool requiredzTBB threadpool requiredGNUzGNU OpenMP only testswindarwinlcCst||SN)nponesnvrZ/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/numba/tests/test_parallel_backend.pyfooGsrcCs2tt||ft||f}|t||Sr)rdotrarange)rrxrrrlinalgLs r cCst||Sr)rrrrrrr PscCs||Srr)abrrr ufunc_fooUsr#cCs|||d<dS)Nrr)r!r"outrrr gufunc_fooYsr%c@seZdZddZdS)runnablecKs ||_dSr)_options)selfoptionsrrr__init__^szrunnable.__init__N)__name__ __module__ __qualname__r*rrrrr&]sr&c@seZdZddZdS) jit_runnercCs>tf|jt}d}d}t||}|||}tj||dSN )rr'rrtestingassert_allcloser(cfuncr!r"expectedgotrrr__call__ds   zjit_runner.__call__Nr+r,r-r8rrrrr.bsr.c@seZdZddZddZdS) mask_runnercKs||_||_dSr)runnermask)r(r;r<r)rrrr*nszmask_runner.__init__cCs|jrt|j|dSr)r<rr;)r(rrrr8rs zmask_runner.__call__Nr+r,r-r*r8rrrrr:msr:c@seZdZddZdS) linalg_runnercCs>tf|jt}d}d}t||}|||}tj||dSr/)rr'r rr2r3r4rrrr8|s   zlinalg_runner.__call__Nr9rrrrr>zsr>c@seZdZddZdS)vectorize_runnercCsRtdgf|jt}tjdtj}}t||}|||}tj||dS)Nz(f4, f4)r1) rr'r#rrandomastypefloat32r2r3r4rrrr8s   zvectorize_runner.__call__Nr9rrrrr?sr?c@seZdZddZdS)guvectorize_runnercCsXdg}t|df|jt}tjdtj}}t||}|||}tj ||dS)Nz(f4, f4, f4[:])z (),()->()r1) rr'r%rr@rArBr#r2r3)r(sigr5r!r"r6r7rrrr8s   zguvectorize_runner.__call__Nr9rrrrrCsrCc Ksr|d}z6tttt|dD]}t|}|q(Wn,tk rl}z| |W5d}~XYnXdS)Nqueueg?) get faulthandlerenablerangeintlenr@choice Exceptionput)fnlistkwargsq_fnerrrchoosers   rUcsfdd}|S)Ncs}d|ifddtdD}|D] }|q,|D] }|q>|sg}|sr||dqXd}t|ddd|DdS) NrEcsg|]}tfdqS))targetargsrP)rU).0i)rOkwsparallel_classrr sz8compile_factory..run_compile..r0Fz)Error(s) occurred in delegated runner: %s cSsg|] }t|qSr)repr)rXrrrrr\s)rIstartjoinemptyappendrF RuntimeError)rOrQZthstherrors_msgr[ queue_impl)rOrZr run_compiles  z$compile_factory..run_compiler)r[rhrirrgrcompile_factorysrjc@seZdZddZddZdS)_proc_class_implcCs ||_dSr)_method)r(methodrrrr*sz_proc_class_impl.__init__cOst|j}|j||Sr)multiprocessing get_contextrlProcess)r(rWrPctxrrrr8s z_proc_class_impl.__call__Nr=rrrrrksrkcCs,|dkr d}t|}t|}|j}||fS)Ndefault)rnrorkQueue)rmrqprocrErrr_get_mp_classess  ruspawnfork forkserverrrc@sjeZdZdZeddedddedddeddedddeddedddeddddeddedddeddddg Ze seddd edddd eddd edddd gZ e e e j d krgZnd d gZgZeD]ZeD]Zeeeeqqd dgZeder$ededede d gedddgedddgeedZddhZdddZdS)TestParallelBackendBasez6 Base class for testing the parallel backends T)nopython)rzcache)rzZnogilparallel)rzrV)rzrVr{)rzr|)rzr|r{ threadingr@multiprocessing_spawnmultiprocessing_forkmultiprocessing_forkserver)Zconcurrent_jitZconcurrent_vectorizeZconcurrent_guvectorizeZconcurrent_mix_useZconcurrent_mix_use_masksomptbbc Cst|jj|_td|j|dkr.t|n|dkr@t|n|dkrRt|nv|dkrdt|nd|dkrvt |nR|dkrttg}t r| t| tt ||D] }||qn td|W5QRXdS) NZ CACHE_DIRrrrrZmultiprocessing_defaultr@zUnknown parallelism supplied %s)r __class__r+Z _cache_dirr thread_implfork_proc_implforkserver_proc_implspawn_proc_impldefault_proc_impl _HAVE_OS_FORKrbr@shuffle ValueError)r(rO parallelismZpsimplrrrris.         z#TestParallelBackendBase.run_compileN)r)r+r,r-__doc__r.r>r?rCZ all_impls_parfors_unsupportedZ parfor_implsextendr NUMBA_NUM_THREADSmasksZ mask_implsrr<rbr:rrrunnersZ safe_backendsrirrrrrysV                   ry)rr workqueuezThreading layer not explicitc@seZdZdZeddZdS)TestParallelBackendav These are like the numba.tests.test_threadsafety tests but designed instead to torture the parallel backend. If a suitable backend is supplied via NUMBA_THREADING_LAYER these tests can be run directly. This test class cannot be run using the multiprocessing option to the test runner (i.e. `./runtests -m`) as daemon processes cannot have children. cCsX|jD]L}|jD]<\}}d|d|}dd}|||}||_t|||qqdS)Ntest_rRcsfdd}|S)Ncs0t}|jrd}||n|jddS)Nz)daemonized processes cannot have children)r)rncurrent_processdaemonskipTestri)r(Zselfprocrfrprr test_methodIs  zBTestParallelBackend.generate..methgen..test_methodr)rrrrrrmethgenHsz-TestParallelBackend.generate..methgen)rritemsr+setattr)clsrnamermethnamerrSrrrgenerateBs  zTestParallelBackend.generateN)r+r,r-r classmethodrrrrrr5s rc@s0eZdZeeedddZddZddZ dS) TestInSubprocessFrrrcCs|tj|tjtj|d}tt|j}zH|| \}}|j dkrZt d|j | f| | fWS|XdS)Nstdoutstderrenvr/process failed with code %s: stderr follows %s )NN subprocessPopenPIPErTimer _TEST_TIMEOUTkillcancelr_ communicate returncodeAssertionErrordecoder(cmdlinerpopentimeoutr$errrrrrun_cmd_s$    zTestInSubprocess.run_cmdcCs0tj}t||d<tjdd|g}|||S)NNUMBA_THREADING_LAYERz-mznumba.runtests)osenvironcopystrsys executabler)r(testZthreading_layerZenv_copyrrrrrun_test_in_separate_processrs  z-TestInSubprocess.run_test_in_separate_processN) r+r,r- skip_no_tbb skip_no_ompunittestskipIfbackendsrrrrrrrZs  rc@s,eZdZdZdZeddZeddZdS)TestSpecificBackenda This is quite contrived, for each test in the TestParallelBackend tests it generates a test that will run the TestParallelBackend test in a new python process with an environment modified to ensure a specific threadsafe backend is used. This is with view of testing the backends independently and in an isolated manner such that if they hang/crash/have issues, it doesn't kill the test suite. Fc sb|j}tj}d|d|}d|||ffdd}d||f} t|| td||dS)NrrRz%s.%s.%scsR|\}}|jr&td||f|d||d|k|d|kdS)Nzstdout: "%s" stderr: "%s"OKFAILERROR)r_DEBUGprintassertIn assertTrue)r(orTbackendZinjected_methodrr test_templates  z2TestSpecificBackend._inject..test_templatez test_%s_%s_%sZ long_running)r,rr+rr ) rrrr backend_guardZthemodZtheclsrr injected_testrrr_injectszTestSpecificBackend._injectc Cst|jD]d\}}|jD]T}|jD]D}|dkrH|dkrHtjdrHq&|dkrZ|dkrZq&|||||q&qq dS)N)rr@rlinux)rr@r) rrrrkeysrplatform startswithr)rrrrrrrrrs  zTestSpecificBackend.generateN)r+r,r-rrrrrrrrrrys  rc@s6eZdZdZdejeZddeiZ dddZ dS) ThreadLayerTestHelperzP Helper class for running an isolated piece of code based on a template z%raif 1: import sys sys.path.insert(0, "%(here)r") import multiprocessing import numpy as np from numba import njit import numba try: import threading_backend_usecases except ImportError as e: print("DEBUG:", sys.path) raise e import os sigterm_handler = threading_backend_usecases.sigterm_handler busy_func = threading_backend_usecases.busy_func def the_test(): %%s if __name__ == "__main__": the_test() hereNcCs|dkrtj}td|d<tj|tjtj|d}tt |j }z8| | \}}|jdkrxtd|j|fW5| X||fS)Nrrrrr)rrrrrrrrrrrrr_rrrrrrrrrs(      zThreadLayerTestHelper.run_cmd)N) r+r,r-rrpathdirname__file__Z_heretemplaterrrrrrs rc@s@eZdZdZdZeeedddZ e ddZ e ddZ d S) TestThreadingLayerSelectionz@ Checks that numba.threading_layer() reports correctly. Frrcs0fdd}d}t||td||dS)NcsZd}|j|}tjd|g}tj}t|d<|j||d\}}|jrVt ||dS)Nzif 1: X = np.arange(1000000.) Y = np.arange(1000000.) Z = busy_func(X, Y) assert numba.threading_layer() == '%s' -crr) rrrrrrrrrr)r(bodyrunmerrr$rrrrrs   z:TestThreadingLayerSelection._inject..test_templatez test_threading_layer_selector_%sZ important)rr )rrrrrrrrrs  z#TestThreadingLayerSelection._injectcCs$|jD]\}}|||q dSr)rrr)rrrrrrrsz$TestThreadingLayerSelection.generateN) r+r,r-rrrrrrrrrrrrrrrs  rc@sZeZdZedddZeeddZeeddZedd Z ed d Z d d Z dS)TestThreadingLayerPriority)env_varcCsJtj}d|d<||d<d|d}tjdt|g}|j||ddS) zJTest setting priority via env var NUMBA_THREADING_LAYER_PRIORITY. rrrZNUMBA_THREADING_LAYER_PRIORITYa import numba # trigger threading layer decision # hence catching invalid THREADING_LAYER_PRIORITY @numba.jit( 'float64[::1](float64[::1], float64[::1])', nopython=True, parallel=True, ) def plus(x, y): return x + y captured_envvar = list("a ".split()) assert numba.config.THREADING_LAYER_PRIORITY == captured_envvar, "priority mismatch" assert numba.threading_layer() == captured_envvar[0], "selected backend mismatch" rrN)rrrrrtextwrapdedentr)r(rrcodecmdrrr each_env_vars  z'TestThreadingLayerPriority.each_env_varcCs2dddg}t|D]}d|}||qdS)Nrrr ) itertools permutationsr`r)r(rrrrrrrtest_valid_env_var2s  z-TestThreadingLayerPriority.test_valid_env_varc CsFd}|t}||W5QRXdD]}||t|jq(dS)Nztbb omp workqueue notvalidhere)z!THREADING_LAYER_PRIORITY invalid:zIt must be a permutation of) assertRaisesrrrr exception)r(rZraisesmsgrrrtest_invalid_env_var:s  z/TestThreadingLayerPriority.test_invalid_env_varcCsdD]}||qdS)N)zomp tbb workqueuezomp workqueue tbbrr(rrrrtest_ompFsz#TestThreadingLayerPriority.test_ompcCsdD]}||qdS)N)ztbb omp workqueueztbb workqueue omprrrrrtest_tbbKsz#TestThreadingLayerPriority.test_tbbcCsdD]}||qdS)N)zworkqueue tbb ompzworkqueue omp tbbrrrrrtest_workqueuePsz)TestThreadingLayerPriority.test_workqueueN) r+r,r-rrrrrrrrrrrrrrs!    rc@sHeZdZdZdZeddZeddZddZ e e d d d Z d S) TestMiscBackendIssueszL Checks fixes for the issues with threading backends implementation FcCs<d}tjd|g}tj}d|d<d|d<|j||ddS) z8 Tests that OMP does not overflow stack aif 1: from numba import vectorize, threading_layer import numpy as np @vectorize(['f4(f4,f4,f4,f4,f4,f4,f4,f4)'], target='parallel') def foo(a, b, c, d, e, f, g, h): return a+b+c+d+e+f+g+h x = np.ones(2**20, np.float32) foo(*([x]*8)) assert threading_layer() == "omp", "omp not found" rrrZ100KZ OMP_STACKSIZErNrrrrrrr(rrrrrrtest_omp_stack_overflow\s   z-TestMiscBackendIssues.test_omp_stack_overflowcCs<d}tjd|g}tj}d|d<d|d<|j||ddS) zq Tests that TBB works well with single thread https://github.com/numba/numba/issues/3440 aMif 1: from numba import njit, prange, threading_layer @njit(parallel=True) def foo(n): acc = 0 for i in prange(n): acc += i return acc foo(100) assert threading_layer() == "tbb", "tbb not found" rrr1rrNrrrrrtest_single_thread_tbbss   z,TestMiscBackendIssues.test_single_thread_tbbc Csd}tjd|g}tj}d|d<d|d<z|j||d\}}WnVtk r}z8|jrbt||t |}| d|d }| ||W5d }~XYnXd S) zW Tests workqueue raises sigabrt if a nested parallel call is performed aif 1: from numba import njit, prange import numpy as np @njit(parallel=True) def nested(x): for i in prange(len(x)): x[i] += 1 @njit(parallel=True) def main(): Z = np.zeros((5, 10)) for i in prange(Z.shape[0]): nested(Z[i]) return Z main() rrr4rrzfailed with codezTNumba workqueue threading layer is terminating: Concurrent access has been detected.N) rrrrrrrrrrr) r(rrrr$rrTZe_msgr6rrr+test_workqueue_aborts_on_nested_parallelisms    zATestMiscBackendIssues.test_workqueue_aborts_on_nested_parallelismzTest needs fork(2)cCs<d}tjd|g}tj}d|d<d|d<|j||ddS)Naif 1: from numba import njit, prange, threading_layer import numpy as np import multiprocessing if __name__ == "__main__": # Need for force fork context (OSX default is "spawn") multiprocessing.set_start_method('fork') @njit(parallel=True) def func(x): return 10. * x arr = np.arange(2.) # run in single process to start Numba's thread pool np.testing.assert_allclose(func(arr), func.py_func(arr)) # now run in a multiprocessing pool to get a fork from a # non-main thread with multiprocessing.Pool(10) as p: result = p.map(func, [arr]) np.testing.assert_allclose(result, func.py_func(np.expand_dims(arr, 0))) assert threading_layer() == "workqueue" rrrrrrrrrrr0test_workqueue_handles_fork_from_non_main_threads   zFTestMiscBackendIssues.test_workqueue_handles_fork_from_non_main_threadN)r+r,r-rrrrrrr r skipUnlessrr rrrrrUs  ) rc@sTeZdZdZdZddZddZddZed d Z d d Z d dZ eddZ dS)TestForkSafetyIssueszV Checks Numba's behaviour in various situations involving GNU OpenMP and fork FcCs"d}tjd|g}||\}}dS)Nzsif 1: from numba.np.ufunc import omppool assert omppool.openmp_vendor == 'GNU' r)rrr)r(rrr$rrrr!test_check_threading_layer_is_gnus z6TestForkSafetyIssues.test_check_threading_layer_is_gnuc Csdd}|j|}tjd|g}z||\}}Wn2tk r^}z|dt|W5d}~XYnXdS)z~ Whilst normally valid, this actually isn't for Numba invariant of OpenMP Checks SIGABRT is received. zif 1: X = np.arange(1000000.) Y = np.arange(1000000.) Z = busy_func(X, Y) pid = os.fork() if pid == 0: Z = busy_func(X, Y) else: os.wait() rzfailed with code -6N)rrrrrrr)r(rrrr$rrTrrr!test_par_parent_os_fork_par_childs  z6TestForkSafetyIssues.test_par_parent_os_fork_par_childcCs<d}|j|}tjd|g}||\}}|jr8t||dS)au Implicit use of multiprocessing fork context. Does this: 1. Start with OpenMP 2. Fork to processes using OpenMP (this is invalid) 3. Joins fork 4. Check the exception pushed onto the queue that is a result of catching SIGTERM coming from the C++ aborting on illegal fork pattern for GNU OpenMP aif 1: mp = multiprocessing.get_context('fork') X = np.arange(1000000.) Y = np.arange(1000000.) q = mp.Queue() # Start OpenMP runtime on parent via parallel function Z = busy_func(X, Y, q) # fork() underneath with no exec, will abort proc = mp.Process(target = busy_func, args=(X, Y, q)) proc.start() err = q.get() assert "Caught SIGTERM" in str(err) rNrrrrrrr(rrrr$rrrr*test_par_parent_implicit_mp_fork_par_childs   z?TestForkSafetyIssues.test_par_parent_implicit_mp_fork_par_childcCs<d}|j|}tjd|g}||\}}|jr8t||dS)au Explicit use of multiprocessing fork context. Does this: 1. Start with OpenMP 2. Fork to processes using OpenMP (this is invalid) 3. Joins fork 4. Check the exception pushed onto the queue that is a result of catching SIGTERM coming from the C++ aborting on illegal fork pattern for GNU OpenMP aif 1: X = np.arange(1000000.) Y = np.arange(1000000.) q = multiprocessing.Queue() # Start OpenMP runtime on parent via parallel function Z = busy_func(X, Y, q) # fork() underneath with no exec, will abort ctx = multiprocessing.get_context('fork') proc = ctx.Process(target = busy_func, args=(X, Y, q)) proc.start() proc.join() err = q.get() assert "Caught SIGTERM" in str(err) rNrrrrr*test_par_parent_explicit_mp_fork_par_child's   z?TestForkSafetyIssues.test_par_parent_explicit_mp_fork_par_childcCs<d}|j|}tjd|g}||\}}|jr8t||dS)z Explicit use of multiprocessing spawn, this is safe. Does this: 1. Start with OpenMP 2. Spawn to processes using OpenMP 3. Join spawns 4. Run some more OpenMP aif 1: X = np.arange(1000000.) Y = np.arange(1000000.) q = multiprocessing.Queue() # Start OpenMP runtime and run on parent via parallel function Z = busy_func(X, Y, q) procs = [] ctx = multiprocessing.get_context('spawn') for x in range(20): # start a lot to try and get overlap ## fork() + exec() to run some OpenMP on children proc = ctx.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) sys.stdout.flush() sys.stderr.flush() proc.start() [p.join() for p in procs] try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") # Run some more OpenMP on parent Z = busy_func(X, Y, q) rNrrrrr-test_par_parent_mp_spawn_par_child_par_parentJs   zBTestForkSafetyIssues.test_par_parent_mp_spawn_par_child_par_parentcCs<d}|j|}tjd|g}||\}}|jr8t||dS)a Implicit use of multiprocessing (will be fork, but cannot declare that in Py2.7 as there's no process launch context). Does this: 1. Start with no OpenMP 2. Fork to processes using OpenMP 3. Join forks 4. Run some OpenMP a,if 1: X = np.arange(1000000.) Y = np.arange(1000000.) q = multiprocessing.Queue() # this is ok procs = [] for x in range(10): # fork() underneath with but no OpenMP in parent, this is ok proc = multiprocessing.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) proc.start() [p.join() for p in procs] # and this is still ok as the OpenMP happened in forks Z = busy_func(X, Y, q) try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") rNrrrrr=test_serial_parent_implicit_mp_fork_par_child_then_par_parentvs   zRTestForkSafetyIssues.test_serial_parent_implicit_mp_fork_par_child_then_par_parentcCs<d}|j|}tjd|g}||\}}|jr8t||dS)z Explicit use of multiprocessing 'fork'. Does this: 1. Start with no OpenMP 2. Fork to processes using OpenMP 3. Join forks 4. Run some OpenMP a'if 1: X = np.arange(1000000.) Y = np.arange(1000000.) q = multiprocessing.Queue() # this is ok procs = [] ctx = multiprocessing.get_context('fork') for x in range(10): # fork() underneath with but no OpenMP in parent, this is ok proc = ctx.Process(target = busy_func, args=(X, Y, q)) procs.append(proc) proc.start() [p.join() for p in procs] # and this is still ok as the OpenMP happened in forks Z = busy_func(X, Y, q) try: q.get(False) except multiprocessing.queues.Empty: pass else: raise RuntimeError("Queue was not empty") rNrrrrr=test_serial_parent_explicit_mp_fork_par_child_then_par_parents   zRTestForkSafetyIssues.test_serial_parent_explicit_mp_fork_par_child_then_par_parentN) r+r,r-rrr rrr rrrrrrrrr s! ",)r c@s(eZdZdZeddZeddZdS)TestTBBSpecificIssuesFcCsLd}tjd|g}||\}}d}||||jrHtd|td|dS)Na%if 1: import threading import numba numba.config.THREADING_LAYER='tbb' from numba import njit, prange, objmode from numba.core.serialize import PickleCallableByPath import os e_running = threading.Event() e_proceed = threading.Event() def indirect_core(): e_running.set() # wait for forker() to have forked while not e_proceed.isSet(): pass indirect = PickleCallableByPath(indirect_core) @njit def obj_mode_func(): with objmode(): indirect() @njit(parallel=True, nogil=True) def work(): acc = 0 for x in prange(10): acc += x obj_mode_func() return acc def runner(): work() def forker(): # wait for the jit function to say it's running while not e_running.isSet(): pass # then fork os.fork() # now fork is done signal the runner to proceed to exit e_proceed.set() numba_runner = threading.Thread(target=runner,) fork_runner = threading.Thread(target=forker,) threads = (numba_runner, fork_runner) for t in threads: t.start() for t in threads: t.join() rz9Attempted to fork from a non-main thread, the TBB libraryOUT:ERR:)rrrrrr)r(rrr$rZmsg_headrrrtest_fork_from_non_main_threads6   z4TestTBBSpecificIssues.test_fork_from_non_main_threadcCs|d}dd|}tjd|g}tj}d|d<|j||d\}}||kr\|dn | d ||j rt d |t d |dS) NzSKIP: COMPILATION FAILEDax if 1: import ctypes import sys import multiprocessing as mp from tempfile import TemporaryDirectory, NamedTemporaryFile from numba.pycc.platform import Toolchain, external_compiler_works from numba import njit, prange, threading_layer import faulthandler faulthandler.enable() if not external_compiler_works(): raise AssertionError('External compilers are not found.') with TemporaryDirectory() as tmpdir: with NamedTemporaryFile(dir=tmpdir) as tmpfile: try: src = """ #define TBB_PREVIEW_WAITING_FOR_WORKERS 1 #include static tbb::task_scheduler_handle tsh; extern "C" { void launch(void) { tsh = tbb::task_scheduler_handle::get(); } } """ cxxfile = f"{tmpfile.name}.cxx" with open(cxxfile, 'wt') as f: f.write(src) tc = Toolchain() object_files = tc.compile_objects([cxxfile,], output_dir=tmpdir) dso_name = f"{tmpfile.name}.so" tc.link_shared(dso_name, object_files, libraries=['tbb',], export_symbols=['launch']) # Load into the process, it doesn't matter whether the # DSO exists on disk once it's loaded in. DLL = ctypes.CDLL(dso_name) except Exception as e: # Something is broken in compilation, could be one of # many things including, but not limited to: missing tbb # headers, incorrect permissions, compilers that don't # work for the above print(e) print('BROKEN_COMPILERS') sys.exit(0) # Do the test, launch this library and also execute a # function with the TBB threading layer. DLL.launch() @njit(parallel=True) def foo(n): acc = 0 for i in prange(n): acc += i return acc foo(1) # Check the threading layer used was TBB assert threading_layer() == 'tbb' # Use mp context for a controlled version of fork, this triggers the # reported bug. ctx = mp.get_context('fork') def nowork(): pass p = ctx.Process(target=nowork) p.start() p.join(10) print("SUCCESS") BROKEN_COMPILERSrrrrz3Compilation of DSO failed. Check output for detailsSUCCESSrr) Zskip_if_no_external_compilerreplacerrrrrrrrrr)r(rrrrr$rrrr&test_lifetime_of_task_scheduler_handle*s KM     z Checks the OpenMP vendor strings are correct ZMSwin32ZIntelrrrN)dictrrrr assertEqualr openmp_vendor)r(r6krrr test_vendorss  zTestOpenMPVendors.test_vendorsN)r+r,r-r)rrrrr#sr#__main__)brrGrrnrr@rrrrrnumpyrZnumbarrrrZnumba.tests.supportrrrr r r rEZt_queueZnumba.testing.mainr Z_RUNNER_TIMEOUTZ numba.corer rZnumba.np.ufunc.parallelrZnumba.np.ufuncrZ_HAVE_TBB_POOL ImportErrorrZ_HAVE_OMP_POOLZscipy.linalg.cython_lapackZscipyZ _HAVE_LAPACKr rrr'Z_gnuompZskip_unless_gnu_omprr_windowsZ_osxmaxsizeZ_32bitrrrr r#r%objectr&r.r:r>r?rCrUrjThreadZ _thread_classrkrursrrrrrryZTHREADING_LAYERZ_specific_backendsrrrrrrrrr rrr#r+mainrrrrs                            X  !36%F hC<