<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en-us" xml:lang="en-us"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta> <meta http-equiv="X-UA-Compatible" content="IE=edge"></meta> <meta name="copyright" content="(C) Copyright 2005"></meta> <meta name="DC.rights.owner" content="(C) Copyright 2005"></meta> <meta name="DC.Type" content="cuda_reference"></meta> <meta name="DC.Title" content="Stream synchronization behavior"></meta> <meta name="DC.Format" content="XHTML"></meta> <meta name="DC.Identifier" content="stream-sync-behavior"></meta> <link rel="stylesheet" type="text/css" href="../common/formatting/commonltr.css"></link> <link rel="stylesheet" type="text/css" href="../common/formatting/site.css"></link> <title>CUDA Runtime API :: CUDA Toolkit Documentation</title> <!--[if lt IE 9]> <script src="../common/formatting/html5shiv-printshiv.min.js"></script> <![endif]--> <script type="text/javascript" charset="utf-8" src="//assets.adobedtm.com/b92787824f2e0e9b68dc2e993f9bd995339fe417/satelliteLib-7ba51e58dc61bcb0e9311aadd02a0108ab24cc6c.js"></script> <script type="text/javascript" charset="utf-8" src="../common/formatting/jquery.min.js"></script> <script type="text/javascript" charset="utf-8" src="../common/formatting/jquery.ba-hashchange.min.js"></script> <script type="text/javascript" charset="utf-8" src="../common/formatting/jquery.scrollintoview.min.js"></script> <script type="text/javascript" src="../search/htmlFileList.js"></script> <script type="text/javascript" src="../search/htmlFileInfoList.js"></script> <script type="text/javascript" src="../search/nwSearchFnt.min.js"></script> <script type="text/javascript" src="../search/stemmers/en_stemmer.min.js"></script> <script type="text/javascript" src="../search/index-1.js"></script> <script type="text/javascript" src="../search/index-2.js"></script> <script type="text/javascript" src="../search/index-3.js"></script> <link rel="canonical" href="http://docs.nvidia.com/cuda/cuda-runtime-api/index.html"></link> <link rel="stylesheet" type="text/css" href="../common/formatting/qwcode.highlight.css"></link> </head> <body> <header id="header"><span id="company">NVIDIA</span><span id="site-title">CUDA Toolkit Documentation</span><form id="search" method="get" action="search"> <input type="text" name="search-text"></input><fieldset id="search-location"> <legend>Search In:</legend> <label><input type="radio" name="search-type" value="site"></input>Entire Site</label> <label><input type="radio" name="search-type" value="document"></input>Just This Document</label></fieldset> <button type="reset">clear search</button> <button id="submit" type="submit">search</button></form> </header> <div id="site-content"> <nav id="site-nav"> <div class="category closed"><a href="../index.html" title="The root of the site.">CUDA Toolkit v10.1.168</a></div> <div class="category"><a href="index.html" title="CUDA Runtime API">CUDA Runtime API</a></div> <ul> <li> <div class="section-link"><a href="driver-vs-runtime-api.html#driver-vs-runtime-api">1. Difference between the driver and runtime APIs </a></div> </li> <li> <div class="section-link"><a href="api-sync-behavior.html#api-sync-behavior">2. API synchronization behavior </a></div> </li> <li> <div class="section-link"><a href="stream-sync-behavior.html#stream-sync-behavior">3. Stream synchronization behavior</a></div> </li> <li> <div class="section-link"><a href="graphs-thread-safety.html#graphs-thread-safety">4. Graph object thread safety</a></div> </li> <li> <div class="section-link"><a href="modules.html#modules">5. Modules</a></div> <ul> <li> <div class="section-link"><a href="group__CUDART__DEVICE.html#group__CUDART__DEVICE">5.1. Device Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__THREAD__DEPRECATED.html#group__CUDART__THREAD__DEPRECATED">5.2. Thread Management [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__ERROR.html#group__CUDART__ERROR">5.3. Error Handling</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__STREAM.html#group__CUDART__STREAM">5.4. Stream Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__EVENT.html#group__CUDART__EVENT">5.5. Event Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__EXTRES__INTEROP.html#group__CUDART__EXTRES__INTEROP">5.6. External Resource Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__EXECUTION.html#group__CUDART__EXECUTION">5.7. Execution Control</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__OCCUPANCY.html#group__CUDART__OCCUPANCY">5.8. Occupancy</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__MEMORY.html#group__CUDART__MEMORY">5.9. Memory Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__MEMORY__DEPRECATED.html#group__CUDART__MEMORY__DEPRECATED">5.10. Memory Management [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__UNIFIED.html#group__CUDART__UNIFIED">5.11. Unified Addressing</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__PEER.html#group__CUDART__PEER">5.12. Peer Device Memory Access</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__OPENGL.html#group__CUDART__OPENGL">5.13. OpenGL Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__OPENGL__DEPRECATED.html#group__CUDART__OPENGL__DEPRECATED">5.14. OpenGL Interoperability [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D9.html#group__CUDART__D3D9">5.15. Direct3D 9 Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D9__DEPRECATED.html#group__CUDART__D3D9__DEPRECATED">5.16. Direct3D 9 Interoperability [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D10.html#group__CUDART__D3D10">5.17. Direct3D 10 Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D10__DEPRECATED.html#group__CUDART__D3D10__DEPRECATED">5.18. Direct3D 10 Interoperability [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D11.html#group__CUDART__D3D11">5.19. Direct3D 11 Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__D3D11__DEPRECATED.html#group__CUDART__D3D11__DEPRECATED">5.20. Direct3D 11 Interoperability [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__VDPAU.html#group__CUDART__VDPAU">5.21. VDPAU Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__EGL.html#group__CUDART__EGL">5.22. EGL Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__INTEROP.html#group__CUDART__INTEROP">5.23. Graphics Interoperability</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__TEXTURE.html#group__CUDART__TEXTURE">5.24. Texture Reference Management [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__SURFACE.html#group__CUDART__SURFACE">5.25. Surface Reference Management [DEPRECATED]</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__TEXTURE__OBJECT.html#group__CUDART__TEXTURE__OBJECT">5.26. Texture Object Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__SURFACE__OBJECT.html#group__CUDART__SURFACE__OBJECT">5.27. Surface Object Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART____VERSION.html#group__CUDART____VERSION">5.28. Version Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__GRAPH.html#group__CUDART__GRAPH">5.29. Graph Management</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__HIGHLEVEL.html#group__CUDART__HIGHLEVEL">5.30. C++ API Routines</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__DRIVER.html#group__CUDART__DRIVER">5.31. Interactions with the CUDA Driver API</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__PROFILER.html#group__CUDART__PROFILER">5.32. Profiler Control</a></div> </li> <li> <div class="section-link"><a href="group__CUDART__TYPES.html#group__CUDART__TYPES">5.33. Data types used by CUDA Runtime</a></div> </li> </ul> </li> <li> <div class="section-link"><a href="annotated.html#annotated">6. Data Structures</a></div> <ul> <li> <div class="section-link"><a href="class____cudaOccupancyB2DHelper.html#class____cudaOccupancyB2DHelper">6.1. __cudaOccupancyB2DHelper</a></div> </li> <li> <div class="section-link"><a href="structcudaChannelFormatDesc.html#structcudaChannelFormatDesc">6.2. cudaChannelFormatDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaDeviceProp.html#structcudaDeviceProp">6.3. cudaDeviceProp</a></div> </li> <li> <div class="section-link"><a href="structcudaEglFrame.html#structcudaEglFrame">6.4. cudaEglFrame</a></div> </li> <li> <div class="section-link"><a href="structcudaEglPlaneDesc.html#structcudaEglPlaneDesc">6.5. cudaEglPlaneDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaExtent.html#structcudaExtent">6.6. cudaExtent</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalMemoryBufferDesc.html#structcudaExternalMemoryBufferDesc">6.7. cudaExternalMemoryBufferDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalMemoryHandleDesc.html#structcudaExternalMemoryHandleDesc">6.8. cudaExternalMemoryHandleDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalMemoryMipmappedArrayDesc.html#structcudaExternalMemoryMipmappedArrayDesc">6.9. cudaExternalMemoryMipmappedArrayDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalSemaphoreHandleDesc.html#structcudaExternalSemaphoreHandleDesc">6.10. cudaExternalSemaphoreHandleDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalSemaphoreSignalParams.html#structcudaExternalSemaphoreSignalParams">6.11. cudaExternalSemaphoreSignalParams</a></div> </li> <li> <div class="section-link"><a href="structcudaExternalSemaphoreWaitParams.html#structcudaExternalSemaphoreWaitParams">6.12. cudaExternalSemaphoreWaitParams</a></div> </li> <li> <div class="section-link"><a href="structcudaFuncAttributes.html#structcudaFuncAttributes">6.13. cudaFuncAttributes</a></div> </li> <li> <div class="section-link"><a href="structcudaHostNodeParams.html#structcudaHostNodeParams">6.14. cudaHostNodeParams</a></div> </li> <li> <div class="section-link"><a href="structcudaIpcEventHandle__t.html#structcudaIpcEventHandle__t">6.15. cudaIpcEventHandle_t</a></div> </li> <li> <div class="section-link"><a href="structcudaIpcMemHandle__t.html#structcudaIpcMemHandle__t">6.16. cudaIpcMemHandle_t</a></div> </li> <li> <div class="section-link"><a href="structcudaKernelNodeParams.html#structcudaKernelNodeParams">6.17. cudaKernelNodeParams</a></div> </li> <li> <div class="section-link"><a href="structcudaLaunchParams.html#structcudaLaunchParams">6.18. cudaLaunchParams</a></div> </li> <li> <div class="section-link"><a href="structcudaMemcpy3DParms.html#structcudaMemcpy3DParms">6.19. cudaMemcpy3DParms</a></div> </li> <li> <div class="section-link"><a href="structcudaMemcpy3DPeerParms.html#structcudaMemcpy3DPeerParms">6.20. cudaMemcpy3DPeerParms</a></div> </li> <li> <div class="section-link"><a href="structcudaMemsetParams.html#structcudaMemsetParams">6.21. cudaMemsetParams</a></div> </li> <li> <div class="section-link"><a href="structcudaPitchedPtr.html#structcudaPitchedPtr">6.22. cudaPitchedPtr</a></div> </li> <li> <div class="section-link"><a href="structcudaPointerAttributes.html#structcudaPointerAttributes">6.23. cudaPointerAttributes</a></div> </li> <li> <div class="section-link"><a href="structcudaPos.html#structcudaPos">6.24. cudaPos</a></div> </li> <li> <div class="section-link"><a href="structcudaResourceDesc.html#structcudaResourceDesc">6.25. cudaResourceDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaResourceViewDesc.html#structcudaResourceViewDesc">6.26. cudaResourceViewDesc</a></div> </li> <li> <div class="section-link"><a href="structcudaTextureDesc.html#structcudaTextureDesc">6.27. cudaTextureDesc</a></div> </li> <li> <div class="section-link"><a href="structCUuuid__st.html#structCUuuid__st">6.28. CUuuid_st</a></div> </li> <li> <div class="section-link"><a href="structsurfaceReference.html#structsurfaceReference">6.29. surfaceReference</a></div> </li> <li> <div class="section-link"><a href="structtextureReference.html#structtextureReference">6.30. textureReference</a></div> </li> </ul> </li> <li> <div class="section-link"><a href="functions.html#functions">7. Data Fields</a></div> </li> <li> <div class="section-link"><a href="deprecated.html#deprecated">8. Deprecated List</a></div> </li> <li> <div class="section-link"><a href="notices-header.html#notices-header">Notices</a></div> <ul></ul> </li> </ul> </nav> <div id="resize-nav"></div> <nav id="search-results"> <h2>Search Results</h2> <ol></ol> </nav> <div id="contents-container"> <div id="breadcrumbs-container"> <div id="breadcrumbs"><a href="api-sync-behavior.html" shape="rect">< Previous</a> | <a href="graphs-thread-safety.html" shape="rect">Next ></a></div> <div id="release-info">CUDA Runtime API (<a href="../../pdf/CUDA_Runtime_API.pdf">PDF</a>) - v10.1.168 (<a href="https://developer.nvidia.com/cuda-toolkit-archive">older</a>) - Last updated April 24, 2019 - <a href="mailto:CUDAIssues@nvidia.com?subject=CUDA Toolkit Documentation Feedback: CUDA Runtime API">Send Feedback</a></div> </div> <article id="contents"> <div class="topic nested1" id="stream-sync-behavior"><a name="stream-sync-behavior" shape="rect"> <!-- --></a><h2 class="topictitle2">3. Stream synchronization behavior</h2> <div class="body refbody"> <div class="section" id="stream-sync-behavior__default-stream"><a name="stream-sync-behavior__default-stream" shape="rect"> <!-- --></a><h2 class="title sectiontitle">Default stream</h2> <p class="p"> The default stream, used when <samp class="ph codeph">0</samp> is passed as a <samp class="ph codeph">cudaStream_t</samp> or by APIs that operate on a stream implicitly, can be configured to have either <a class="xref" href="stream-sync-behavior.html#stream-sync-behavior__legacy-default-stream" shape="rect">legacy</a> or <a class="xref" href="stream-sync-behavior.html#stream-sync-behavior__per-thread-default-stream" shape="rect">per-thread</a> synchronization behavior as described below. </p> <p class="p"> The behavior can be controlled per compilation unit with the <samp class="ph codeph">--default-stream</samp> nvcc option. Alternatively, per-thread behavior can be enabled by defining the <samp class="ph codeph">CUDA_API_PER_THREAD_DEFAULT_STREAM</samp> macro before including any CUDA headers. Either way, the <samp class="ph codeph">CUDA_API_PER_THREAD_DEFAULT_STREAM</samp> macro will be defined in compilation units using per-thread synchronization behavior. </p> </div> <div class="section" id="stream-sync-behavior__legacy-default-stream"><a name="stream-sync-behavior__legacy-default-stream" shape="rect"> <!-- --></a><h2 class="title sectiontitle">Legacy default stream</h2> <p class="p"> The legacy default stream is an implicit stream which synchronizes with all other streams in the same <samp class="ph codeph">CUcontext</samp> except for non-blocking streams, described below. (For applications using the runtime APIs only, there will be one context per device.) When an action is taken in the legacy stream such as a kernel launch or <samp class="ph codeph">cudaStreamWaitEvent()</samp>, the legacy stream first waits on all blocking streams, the action is queued in the legacy stream, and then all blocking streams wait on the legacy stream. </p> <p class="p"> For example, the following code launches a kernel <samp class="ph codeph">k_1</samp> in stream <samp class="ph codeph">s</samp>, then <samp class="ph codeph">k_2</samp> in the legacy stream, then <samp class="ph codeph">k_3</samp> in stream <samp class="ph codeph">s</samp>: </p><pre xml:space="preserve"> k_1<<<1, 1, 0, s>>>(); k_2<<<1, 1>>>(); k_3<<<1, 1, 0, s>>>();</pre><p class="p"> The resulting behavior is that <samp class="ph codeph">k_2</samp> will block on <samp class="ph codeph">k_1</samp> and <samp class="ph codeph">k_3</samp> will block on <samp class="ph codeph">k_2</samp>. </p> <p class="p"> Non-blocking streams which do not synchronize with the legacy stream can be created using the <samp class="ph codeph">cudaStreamNonBlocking</samp> flag with the stream creation APIs. </p> <p class="p"> The legacy default stream can be used explicitly with the <samp class="ph codeph">CUstream</samp> (<samp class="ph codeph">cudaStream_t</samp>) handle <samp class="ph codeph">CU_STREAM_LEGACY</samp> (<samp class="ph codeph">cudaStreamLegacy</samp>). </p> </div> <div class="section" id="stream-sync-behavior__per-thread-default-stream"><a name="stream-sync-behavior__per-thread-default-stream" shape="rect"> <!-- --></a><h2 class="title sectiontitle">Per-thread default stream</h2> <p class="p"> The per-thread default stream is an implicit stream local to both the thread and the <samp class="ph codeph">CUcontext</samp>, and which does not synchronize with other streams (just like explcitly created streams). The per-thread default stream is not a non-blocking stream and will synchronize with the legacy default stream if both are used in a program. </p> <p class="p"> The per-thread default stream can be used explicitly with the <samp class="ph codeph">CUstream</samp> (<samp class="ph codeph">cudaStream_t</samp>) handle <samp class="ph codeph">CU_STREAM_PER_THREAD</samp> (<samp class="ph codeph">cudaStreamPerThread</samp>). </p> </div> </div> </div> <hr id="contents-end"></hr> </article> </div> </div> <script language="JavaScript" type="text/javascript" charset="utf-8" src="../common/formatting/common.min.js"></script> <script language="JavaScript" type="text/javascript" charset="utf-8" src="../common/scripts/google-analytics/google-analytics-write.js"></script> <script language="JavaScript" type="text/javascript" charset="utf-8" src="../common/scripts/google-analytics/google-analytics-tracker.js"></script> <script type="text/javascript">var switchTo5x=true;</script><script type="text/javascript" src="http://w.sharethis.com/button/buttons.js"></script><script type="text/javascript">stLight.options({publisher: "998dc202-a267-4d8e-bce9-14debadb8d92", doNotHash: false, doNotCopy: false, hashAddressBar: false});</script><script type="text/javascript">_satellite.pageBottom();</script></body> </html>