1eda14cbcSMatt Macy /* 2eda14cbcSMatt Macy * CDDL HEADER START 3eda14cbcSMatt Macy * 4eda14cbcSMatt Macy * This file and its contents are supplied under the terms of the 5eda14cbcSMatt Macy * Common Development and Distribution License ("CDDL"), version 1.0. 6eda14cbcSMatt Macy * You may only use this file in accordance with the terms of version 7eda14cbcSMatt Macy * 1.0 of the CDDL. 8eda14cbcSMatt Macy * 9eda14cbcSMatt Macy * A full copy of the text of the CDDL should have accompanied this 10eda14cbcSMatt Macy * source. A copy of the CDDL is also available via the Internet at 11eda14cbcSMatt Macy * http://www.illumos.org/license/CDDL. 12eda14cbcSMatt Macy * 13eda14cbcSMatt Macy * CDDL HEADER END 14eda14cbcSMatt Macy */ 15eda14cbcSMatt Macy 16eda14cbcSMatt Macy /* 17eda14cbcSMatt Macy * Copyright (c) 2017, 2020 by Delphix. All rights reserved. 18eda14cbcSMatt Macy */ 19eda14cbcSMatt Macy 20eda14cbcSMatt Macy /* 21eda14cbcSMatt Macy * ZTHR Infrastructure 22eda14cbcSMatt Macy * =================== 23eda14cbcSMatt Macy * 24eda14cbcSMatt Macy * ZTHR threads are used for isolated operations that span multiple txgs 25eda14cbcSMatt Macy * within a SPA. They generally exist from SPA creation/loading and until 26eda14cbcSMatt Macy * the SPA is exported/destroyed. The ideal requirements for an operation 27eda14cbcSMatt Macy * to be modeled with a zthr are the following: 28eda14cbcSMatt Macy * 29eda14cbcSMatt Macy * 1] The operation needs to run over multiple txgs. 30eda14cbcSMatt Macy * 2] There is be a single point of reference in memory or on disk that 31eda14cbcSMatt Macy * indicates whether the operation should run/is running or has 32eda14cbcSMatt Macy * stopped. 33eda14cbcSMatt Macy * 34eda14cbcSMatt Macy * If the operation satisfies the above then the following rules guarantee 35eda14cbcSMatt Macy * a certain level of correctness: 36eda14cbcSMatt Macy * 37eda14cbcSMatt Macy * 1] Any thread EXCEPT the zthr changes the work indicator from stopped 38eda14cbcSMatt Macy * to running but not the opposite. 39eda14cbcSMatt Macy * 2] Only the zthr can change the work indicator from running to stopped 40eda14cbcSMatt Macy * (e.g. when it is done) but not the opposite. 41eda14cbcSMatt Macy * 42eda14cbcSMatt Macy * This way a normal zthr cycle should go like this: 43eda14cbcSMatt Macy * 44eda14cbcSMatt Macy * 1] An external thread changes the work indicator from stopped to 45eda14cbcSMatt Macy * running and wakes up the zthr. 46eda14cbcSMatt Macy * 2] The zthr wakes up, checks the indicator and starts working. 47eda14cbcSMatt Macy * 3] When the zthr is done, it changes the indicator to stopped, allowing 48eda14cbcSMatt Macy * a new cycle to start. 49eda14cbcSMatt Macy * 50eda14cbcSMatt Macy * Besides being awakened by other threads, a zthr can be configured 51eda14cbcSMatt Macy * during creation to wakeup on its own after a specified interval 52eda14cbcSMatt Macy * [see zthr_create_timer()]. 53eda14cbcSMatt Macy * 54eda14cbcSMatt Macy * Note: ZTHR threads are NOT a replacement for generic threads! Please 55eda14cbcSMatt Macy * ensure that they fit your use-case well before using them. 56eda14cbcSMatt Macy * 57eda14cbcSMatt Macy * == ZTHR creation 58eda14cbcSMatt Macy * 592c48331dSMatt Macy * Every zthr needs four inputs to start running: 60eda14cbcSMatt Macy * 61eda14cbcSMatt Macy * 1] A user-defined checker function (checkfunc) that decides whether 62eda14cbcSMatt Macy * the zthr should start working or go to sleep. The function should 63eda14cbcSMatt Macy * return TRUE when the zthr needs to work or FALSE to let it sleep, 64eda14cbcSMatt Macy * and should adhere to the following signature: 65eda14cbcSMatt Macy * boolean_t checkfunc_name(void *args, zthr_t *t); 66eda14cbcSMatt Macy * 67eda14cbcSMatt Macy * 2] A user-defined ZTHR function (func) which the zthr executes when 68eda14cbcSMatt Macy * it is not sleeping. The function should adhere to the following 69eda14cbcSMatt Macy * signature type: 70eda14cbcSMatt Macy * void func_name(void *args, zthr_t *t); 71eda14cbcSMatt Macy * 72eda14cbcSMatt Macy * 3] A void args pointer that will be passed to checkfunc and func 73eda14cbcSMatt Macy * implicitly by the infrastructure. 74eda14cbcSMatt Macy * 752c48331dSMatt Macy * 4] A name for the thread. This string must be valid for the lifetime 762c48331dSMatt Macy * of the zthr. 772c48331dSMatt Macy * 78eda14cbcSMatt Macy * The reason why the above API needs two different functions, 79eda14cbcSMatt Macy * instead of one that both checks and does the work, has to do with 80eda14cbcSMatt Macy * the zthr's internal state lock (zthr_state_lock) and the allowed 81eda14cbcSMatt Macy * cancellation windows. We want to hold the zthr_state_lock while 82eda14cbcSMatt Macy * running checkfunc but not while running func. This way the zthr 83eda14cbcSMatt Macy * can be cancelled while doing work and not while checking for work. 84eda14cbcSMatt Macy * 85eda14cbcSMatt Macy * To start a zthr: 862faf504dSMartin Matuska * zthr_t *zthr_pointer = zthr_create(checkfunc, func, args, 872faf504dSMartin Matuska * pri); 88eda14cbcSMatt Macy * or 89eda14cbcSMatt Macy * zthr_t *zthr_pointer = zthr_create_timer(checkfunc, func, 902faf504dSMartin Matuska * args, max_sleep, pri); 91eda14cbcSMatt Macy * 92eda14cbcSMatt Macy * After that you should be able to wakeup, cancel, and resume the 93eda14cbcSMatt Macy * zthr from another thread using the zthr_pointer. 94eda14cbcSMatt Macy * 95eda14cbcSMatt Macy * NOTE: ZTHR threads could potentially wake up spuriously and the 96eda14cbcSMatt Macy * user should take this into account when writing a checkfunc. 97eda14cbcSMatt Macy * [see ZTHR state transitions] 98eda14cbcSMatt Macy * 99eda14cbcSMatt Macy * == ZTHR wakeup 100eda14cbcSMatt Macy * 101eda14cbcSMatt Macy * ZTHR wakeup should be used when new work is added for the zthr. The 102eda14cbcSMatt Macy * sleeping zthr will wakeup, see that it has more work to complete 103eda14cbcSMatt Macy * and proceed. This can be invoked from open or syncing context. 104eda14cbcSMatt Macy * 105eda14cbcSMatt Macy * To wakeup a zthr: 106eda14cbcSMatt Macy * zthr_wakeup(zthr_t *t) 107eda14cbcSMatt Macy * 108eda14cbcSMatt Macy * == ZTHR cancellation and resumption 109eda14cbcSMatt Macy * 110eda14cbcSMatt Macy * ZTHR threads must be cancelled when their SPA is being exported 111eda14cbcSMatt Macy * or when they need to be paused so they don't interfere with other 112eda14cbcSMatt Macy * operations. 113eda14cbcSMatt Macy * 114eda14cbcSMatt Macy * To cancel a zthr: 115eda14cbcSMatt Macy * zthr_cancel(zthr_pointer); 116eda14cbcSMatt Macy * 117eda14cbcSMatt Macy * To resume it: 118eda14cbcSMatt Macy * zthr_resume(zthr_pointer); 119eda14cbcSMatt Macy * 120eda14cbcSMatt Macy * ZTHR cancel and resume should be invoked in open context during the 121eda14cbcSMatt Macy * lifecycle of the pool as it is imported, exported or destroyed. 122eda14cbcSMatt Macy * 123eda14cbcSMatt Macy * A zthr will implicitly check if it has received a cancellation 124eda14cbcSMatt Macy * signal every time func returns and every time it wakes up [see 125eda14cbcSMatt Macy * ZTHR state transitions below]. 126eda14cbcSMatt Macy * 127eda14cbcSMatt Macy * At times, waiting for the zthr's func to finish its job may take 128eda14cbcSMatt Macy * time. This may be very time-consuming for some operations that 129eda14cbcSMatt Macy * need to cancel the SPA's zthrs (e.g spa_export). For this scenario 130eda14cbcSMatt Macy * the user can explicitly make their ZTHR function aware of incoming 131eda14cbcSMatt Macy * cancellation signals using zthr_iscancelled(). A common pattern for 132eda14cbcSMatt Macy * that looks like this: 133eda14cbcSMatt Macy * 134eda14cbcSMatt Macy * int 135eda14cbcSMatt Macy * func_name(void *args, zthr_t *t) 136eda14cbcSMatt Macy * { 137eda14cbcSMatt Macy * ... <unpack args> ... 138eda14cbcSMatt Macy * while (!work_done && !zthr_iscancelled(t)) { 139eda14cbcSMatt Macy * ... <do more work> ... 140eda14cbcSMatt Macy * } 141eda14cbcSMatt Macy * } 142eda14cbcSMatt Macy * 143eda14cbcSMatt Macy * == ZTHR cleanup 144eda14cbcSMatt Macy * 145eda14cbcSMatt Macy * Cancelling a zthr doesn't clean up its metadata (internal locks, 146eda14cbcSMatt Macy * function pointers to func and checkfunc, etc..). This is because 147eda14cbcSMatt Macy * we want to keep them around in case we want to resume the execution 148eda14cbcSMatt Macy * of the zthr later. Similarly for zthrs that exit themselves. 149eda14cbcSMatt Macy * 150eda14cbcSMatt Macy * To completely cleanup a zthr, cancel it first to ensure that it 151eda14cbcSMatt Macy * is not running and then use zthr_destroy(). 152eda14cbcSMatt Macy * 153eda14cbcSMatt Macy * == ZTHR state transitions 154eda14cbcSMatt Macy * 155eda14cbcSMatt Macy * zthr creation 156eda14cbcSMatt Macy * + 157eda14cbcSMatt Macy * | 158eda14cbcSMatt Macy * | woke up 159eda14cbcSMatt Macy * | +--------------+ sleep 160eda14cbcSMatt Macy * | | ^ 161eda14cbcSMatt Macy * | | | 162eda14cbcSMatt Macy * | | | FALSE 163eda14cbcSMatt Macy * | | | 164eda14cbcSMatt Macy * v v FALSE + 165eda14cbcSMatt Macy * cancelled? +---------> checkfunc? 166eda14cbcSMatt Macy * + ^ + 167eda14cbcSMatt Macy * | | | 168eda14cbcSMatt Macy * | | | TRUE 169eda14cbcSMatt Macy * | | | 170eda14cbcSMatt Macy * | | func returned v 171eda14cbcSMatt Macy * | +---------------+ func 172eda14cbcSMatt Macy * | 173eda14cbcSMatt Macy * | TRUE 174eda14cbcSMatt Macy * | 175eda14cbcSMatt Macy * v 176eda14cbcSMatt Macy * zthr stopped running 177eda14cbcSMatt Macy * 178eda14cbcSMatt Macy * == Implementation of ZTHR requests 179eda14cbcSMatt Macy * 180eda14cbcSMatt Macy * ZTHR cancel and resume are requests on a zthr to change its 181eda14cbcSMatt Macy * internal state. These requests are serialized using the 182eda14cbcSMatt Macy * zthr_request_lock, while changes in its internal state are 183eda14cbcSMatt Macy * protected by the zthr_state_lock. A request will first acquire 184eda14cbcSMatt Macy * the zthr_request_lock and then immediately acquire the 185eda14cbcSMatt Macy * zthr_state_lock. We do this so that incoming requests are 186eda14cbcSMatt Macy * serialized using the request lock, while still allowing us 187eda14cbcSMatt Macy * to use the state lock for thread communication via zthr_cv. 188eda14cbcSMatt Macy * 189eda14cbcSMatt Macy * ZTHR wakeup broadcasts to zthr_cv, causing sleeping threads 190eda14cbcSMatt Macy * to wakeup. It acquires the zthr_state_lock but not the 191eda14cbcSMatt Macy * zthr_request_lock, so that a wakeup on a zthr in the middle 192eda14cbcSMatt Macy * of being cancelled will not block. 193eda14cbcSMatt Macy */ 194eda14cbcSMatt Macy 195eda14cbcSMatt Macy #include <sys/zfs_context.h> 196eda14cbcSMatt Macy #include <sys/zthr.h> 197eda14cbcSMatt Macy 198eda14cbcSMatt Macy struct zthr { 199eda14cbcSMatt Macy /* running thread doing the work */ 200eda14cbcSMatt Macy kthread_t *zthr_thread; 201eda14cbcSMatt Macy 202eda14cbcSMatt Macy /* lock protecting internal data & invariants */ 203eda14cbcSMatt Macy kmutex_t zthr_state_lock; 204eda14cbcSMatt Macy 205eda14cbcSMatt Macy /* mutex that serializes external requests */ 206eda14cbcSMatt Macy kmutex_t zthr_request_lock; 207eda14cbcSMatt Macy 208eda14cbcSMatt Macy /* notification mechanism for requests */ 209eda14cbcSMatt Macy kcondvar_t zthr_cv; 210eda14cbcSMatt Macy 211eda14cbcSMatt Macy /* flag set to true if we are canceling the zthr */ 212eda14cbcSMatt Macy boolean_t zthr_cancel; 213eda14cbcSMatt Macy 214eda14cbcSMatt Macy /* flag set to true if we are waiting for the zthr to finish */ 215eda14cbcSMatt Macy boolean_t zthr_haswaiters; 216eda14cbcSMatt Macy kcondvar_t zthr_wait_cv; 217eda14cbcSMatt Macy /* 218eda14cbcSMatt Macy * maximum amount of time that the zthr is spent sleeping; 219eda14cbcSMatt Macy * if this is 0, the thread doesn't wake up until it gets 220eda14cbcSMatt Macy * signaled. 221eda14cbcSMatt Macy */ 222eda14cbcSMatt Macy hrtime_t zthr_sleep_timeout; 223eda14cbcSMatt Macy 2242faf504dSMartin Matuska /* Thread priority */ 2252faf504dSMartin Matuska pri_t zthr_pri; 2262faf504dSMartin Matuska 227eda14cbcSMatt Macy /* consumer-provided callbacks & data */ 228eda14cbcSMatt Macy zthr_checkfunc_t *zthr_checkfunc; 229eda14cbcSMatt Macy zthr_func_t *zthr_func; 230eda14cbcSMatt Macy void *zthr_arg; 2312c48331dSMatt Macy const char *zthr_name; 232eda14cbcSMatt Macy }; 233eda14cbcSMatt Macy 234eda14cbcSMatt Macy static void 235eda14cbcSMatt Macy zthr_procedure(void *arg) 236eda14cbcSMatt Macy { 237eda14cbcSMatt Macy zthr_t *t = arg; 238eda14cbcSMatt Macy 239eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 240eda14cbcSMatt Macy ASSERT3P(t->zthr_thread, ==, curthread); 241eda14cbcSMatt Macy 242eda14cbcSMatt Macy while (!t->zthr_cancel) { 243eda14cbcSMatt Macy if (t->zthr_checkfunc(t->zthr_arg, t)) { 244eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 245eda14cbcSMatt Macy t->zthr_func(t->zthr_arg, t); 246eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 247eda14cbcSMatt Macy } else { 248eda14cbcSMatt Macy if (t->zthr_sleep_timeout == 0) { 2492c48331dSMatt Macy cv_wait_idle(&t->zthr_cv, &t->zthr_state_lock); 250eda14cbcSMatt Macy } else { 2512c48331dSMatt Macy (void) cv_timedwait_idle_hires(&t->zthr_cv, 252eda14cbcSMatt Macy &t->zthr_state_lock, t->zthr_sleep_timeout, 253eda14cbcSMatt Macy MSEC2NSEC(1), 0); 254eda14cbcSMatt Macy } 255eda14cbcSMatt Macy } 256eda14cbcSMatt Macy if (t->zthr_haswaiters) { 257eda14cbcSMatt Macy t->zthr_haswaiters = B_FALSE; 258eda14cbcSMatt Macy cv_broadcast(&t->zthr_wait_cv); 259eda14cbcSMatt Macy } 260eda14cbcSMatt Macy } 261eda14cbcSMatt Macy 262eda14cbcSMatt Macy /* 263eda14cbcSMatt Macy * Clear out the kernel thread metadata and notify the 264eda14cbcSMatt Macy * zthr_cancel() thread that we've stopped running. 265eda14cbcSMatt Macy */ 266eda14cbcSMatt Macy t->zthr_thread = NULL; 267eda14cbcSMatt Macy t->zthr_cancel = B_FALSE; 268eda14cbcSMatt Macy cv_broadcast(&t->zthr_cv); 269eda14cbcSMatt Macy 270eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 271eda14cbcSMatt Macy thread_exit(); 272eda14cbcSMatt Macy } 273eda14cbcSMatt Macy 274eda14cbcSMatt Macy zthr_t * 275eda14cbcSMatt Macy zthr_create(const char *zthr_name, zthr_checkfunc_t *checkfunc, 2762faf504dSMartin Matuska zthr_func_t *func, void *arg, pri_t pri) 277eda14cbcSMatt Macy { 278eda14cbcSMatt Macy return (zthr_create_timer(zthr_name, checkfunc, 2792faf504dSMartin Matuska func, arg, (hrtime_t)0, pri)); 280eda14cbcSMatt Macy } 281eda14cbcSMatt Macy 282eda14cbcSMatt Macy /* 283eda14cbcSMatt Macy * Create a zthr with specified maximum sleep time. If the time 284eda14cbcSMatt Macy * in sleeping state exceeds max_sleep, a wakeup(do the check and 285eda14cbcSMatt Macy * start working if required) will be triggered. 286eda14cbcSMatt Macy */ 287eda14cbcSMatt Macy zthr_t * 288eda14cbcSMatt Macy zthr_create_timer(const char *zthr_name, zthr_checkfunc_t *checkfunc, 2892faf504dSMartin Matuska zthr_func_t *func, void *arg, hrtime_t max_sleep, pri_t pri) 290eda14cbcSMatt Macy { 291eda14cbcSMatt Macy zthr_t *t = kmem_zalloc(sizeof (*t), KM_SLEEP); 292eda14cbcSMatt Macy mutex_init(&t->zthr_state_lock, NULL, MUTEX_DEFAULT, NULL); 293eda14cbcSMatt Macy mutex_init(&t->zthr_request_lock, NULL, MUTEX_DEFAULT, NULL); 294eda14cbcSMatt Macy cv_init(&t->zthr_cv, NULL, CV_DEFAULT, NULL); 295eda14cbcSMatt Macy cv_init(&t->zthr_wait_cv, NULL, CV_DEFAULT, NULL); 296eda14cbcSMatt Macy 297eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 298eda14cbcSMatt Macy t->zthr_checkfunc = checkfunc; 299eda14cbcSMatt Macy t->zthr_func = func; 300eda14cbcSMatt Macy t->zthr_arg = arg; 301eda14cbcSMatt Macy t->zthr_sleep_timeout = max_sleep; 3022c48331dSMatt Macy t->zthr_name = zthr_name; 3032faf504dSMartin Matuska t->zthr_pri = pri; 304eda14cbcSMatt Macy 305eda14cbcSMatt Macy t->zthr_thread = thread_create_named(zthr_name, NULL, 0, 3062faf504dSMartin Matuska zthr_procedure, t, 0, &p0, TS_RUN, pri); 307eda14cbcSMatt Macy 308eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 309eda14cbcSMatt Macy 310eda14cbcSMatt Macy return (t); 311eda14cbcSMatt Macy } 312eda14cbcSMatt Macy 313eda14cbcSMatt Macy void 314eda14cbcSMatt Macy zthr_destroy(zthr_t *t) 315eda14cbcSMatt Macy { 316eda14cbcSMatt Macy ASSERT(!MUTEX_HELD(&t->zthr_state_lock)); 317eda14cbcSMatt Macy ASSERT(!MUTEX_HELD(&t->zthr_request_lock)); 318eda14cbcSMatt Macy VERIFY3P(t->zthr_thread, ==, NULL); 319eda14cbcSMatt Macy mutex_destroy(&t->zthr_request_lock); 320eda14cbcSMatt Macy mutex_destroy(&t->zthr_state_lock); 321eda14cbcSMatt Macy cv_destroy(&t->zthr_cv); 322eda14cbcSMatt Macy cv_destroy(&t->zthr_wait_cv); 323eda14cbcSMatt Macy kmem_free(t, sizeof (*t)); 324eda14cbcSMatt Macy } 325eda14cbcSMatt Macy 326eda14cbcSMatt Macy /* 327eda14cbcSMatt Macy * Wake up the zthr if it is sleeping. If the thread has been cancelled 328eda14cbcSMatt Macy * or is in the process of being cancelled, this is a no-op. 329eda14cbcSMatt Macy */ 330eda14cbcSMatt Macy void 331eda14cbcSMatt Macy zthr_wakeup(zthr_t *t) 332eda14cbcSMatt Macy { 333eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 334eda14cbcSMatt Macy 335eda14cbcSMatt Macy /* 336eda14cbcSMatt Macy * There are 5 states that we can find the zthr when issuing 337eda14cbcSMatt Macy * this broadcast: 338eda14cbcSMatt Macy * 339eda14cbcSMatt Macy * [1] The common case of the thread being asleep, at which 340eda14cbcSMatt Macy * point the broadcast will wake it up. 341eda14cbcSMatt Macy * [2] The thread has been cancelled. Waking up a cancelled 342eda14cbcSMatt Macy * thread is a no-op. Any work that is still left to be 343eda14cbcSMatt Macy * done should be handled the next time the thread is 344eda14cbcSMatt Macy * resumed. 345eda14cbcSMatt Macy * [3] The thread is doing work and is already up, so this 346eda14cbcSMatt Macy * is basically a no-op. 347eda14cbcSMatt Macy * [4] The thread was just created/resumed, in which case the 348eda14cbcSMatt Macy * behavior is similar to [3]. 349eda14cbcSMatt Macy * [5] The thread is in the middle of being cancelled, which 350eda14cbcSMatt Macy * will be a no-op. 351eda14cbcSMatt Macy */ 352eda14cbcSMatt Macy cv_broadcast(&t->zthr_cv); 353eda14cbcSMatt Macy 354eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 355eda14cbcSMatt Macy } 356eda14cbcSMatt Macy 357eda14cbcSMatt Macy /* 358eda14cbcSMatt Macy * Sends a cancel request to the zthr and blocks until the zthr is 359eda14cbcSMatt Macy * cancelled. If the zthr is not running (e.g. has been cancelled 360eda14cbcSMatt Macy * already), this is a no-op. Note that this function should not be 361eda14cbcSMatt Macy * called from syncing context as it could deadlock with the zthr_func. 362eda14cbcSMatt Macy */ 363eda14cbcSMatt Macy void 364eda14cbcSMatt Macy zthr_cancel(zthr_t *t) 365eda14cbcSMatt Macy { 366eda14cbcSMatt Macy mutex_enter(&t->zthr_request_lock); 367eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 368eda14cbcSMatt Macy 369eda14cbcSMatt Macy /* 370eda14cbcSMatt Macy * Since we are holding the zthr_state_lock at this point 371eda14cbcSMatt Macy * we can find the state in one of the following 4 states: 372eda14cbcSMatt Macy * 373eda14cbcSMatt Macy * [1] The thread has already been cancelled, therefore 374eda14cbcSMatt Macy * there is nothing for us to do. 375eda14cbcSMatt Macy * [2] The thread is sleeping so we set the flag, broadcast 376eda14cbcSMatt Macy * the CV and wait for it to exit. 377eda14cbcSMatt Macy * [3] The thread is doing work, in which case we just set 378eda14cbcSMatt Macy * the flag and wait for it to finish. 379eda14cbcSMatt Macy * [4] The thread was just created/resumed, in which case 380eda14cbcSMatt Macy * the behavior is similar to [3]. 381eda14cbcSMatt Macy * 382eda14cbcSMatt Macy * Since requests are serialized, by the time that we get 383eda14cbcSMatt Macy * control back we expect that the zthr is cancelled and 384eda14cbcSMatt Macy * not running anymore. 385eda14cbcSMatt Macy */ 386eda14cbcSMatt Macy if (t->zthr_thread != NULL) { 387eda14cbcSMatt Macy t->zthr_cancel = B_TRUE; 388eda14cbcSMatt Macy 389eda14cbcSMatt Macy /* broadcast in case the zthr is sleeping */ 390eda14cbcSMatt Macy cv_broadcast(&t->zthr_cv); 391eda14cbcSMatt Macy 392eda14cbcSMatt Macy while (t->zthr_thread != NULL) 393eda14cbcSMatt Macy cv_wait(&t->zthr_cv, &t->zthr_state_lock); 394eda14cbcSMatt Macy 395eda14cbcSMatt Macy ASSERT(!t->zthr_cancel); 396eda14cbcSMatt Macy } 397eda14cbcSMatt Macy 398eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 399eda14cbcSMatt Macy mutex_exit(&t->zthr_request_lock); 400eda14cbcSMatt Macy } 401eda14cbcSMatt Macy 402eda14cbcSMatt Macy /* 403eda14cbcSMatt Macy * Sends a resume request to the supplied zthr. If the zthr is already 404eda14cbcSMatt Macy * running this is a no-op. Note that this function should not be 405eda14cbcSMatt Macy * called from syncing context as it could deadlock with the zthr_func. 406eda14cbcSMatt Macy */ 407eda14cbcSMatt Macy void 408eda14cbcSMatt Macy zthr_resume(zthr_t *t) 409eda14cbcSMatt Macy { 410eda14cbcSMatt Macy mutex_enter(&t->zthr_request_lock); 411eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 412eda14cbcSMatt Macy 413eda14cbcSMatt Macy ASSERT3P(&t->zthr_checkfunc, !=, NULL); 414eda14cbcSMatt Macy ASSERT3P(&t->zthr_func, !=, NULL); 415eda14cbcSMatt Macy ASSERT(!t->zthr_cancel); 416eda14cbcSMatt Macy ASSERT(!t->zthr_haswaiters); 417eda14cbcSMatt Macy 418eda14cbcSMatt Macy /* 419eda14cbcSMatt Macy * There are 4 states that we find the zthr in at this point 420eda14cbcSMatt Macy * given the locks that we hold: 421eda14cbcSMatt Macy * 422eda14cbcSMatt Macy * [1] The zthr was cancelled, so we spawn a new thread for 423eda14cbcSMatt Macy * the zthr (common case). 424eda14cbcSMatt Macy * [2] The zthr is running at which point this is a no-op. 425eda14cbcSMatt Macy * [3] The zthr is sleeping at which point this is a no-op. 426eda14cbcSMatt Macy * [4] The zthr was just spawned at which point this is a 427eda14cbcSMatt Macy * no-op. 428eda14cbcSMatt Macy */ 429eda14cbcSMatt Macy if (t->zthr_thread == NULL) { 4302c48331dSMatt Macy t->zthr_thread = thread_create_named(t->zthr_name, NULL, 0, 4312faf504dSMartin Matuska zthr_procedure, t, 0, &p0, TS_RUN, t->zthr_pri); 432eda14cbcSMatt Macy } 433eda14cbcSMatt Macy 434eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 435eda14cbcSMatt Macy mutex_exit(&t->zthr_request_lock); 436eda14cbcSMatt Macy } 437eda14cbcSMatt Macy 438eda14cbcSMatt Macy /* 439eda14cbcSMatt Macy * This function is intended to be used by the zthr itself 440eda14cbcSMatt Macy * (specifically the zthr_func callback provided) to check 441eda14cbcSMatt Macy * if another thread has signaled it to stop running before 442eda14cbcSMatt Macy * doing some expensive operation. 443eda14cbcSMatt Macy * 444eda14cbcSMatt Macy * returns TRUE if we are in the middle of trying to cancel 445eda14cbcSMatt Macy * this thread. 446eda14cbcSMatt Macy * 447eda14cbcSMatt Macy * returns FALSE otherwise. 448eda14cbcSMatt Macy */ 449eda14cbcSMatt Macy boolean_t 450eda14cbcSMatt Macy zthr_iscancelled(zthr_t *t) 451eda14cbcSMatt Macy { 452eda14cbcSMatt Macy ASSERT3P(t->zthr_thread, ==, curthread); 453eda14cbcSMatt Macy 454eda14cbcSMatt Macy /* 455eda14cbcSMatt Macy * The majority of the functions here grab zthr_request_lock 456eda14cbcSMatt Macy * first and then zthr_state_lock. This function only grabs 457eda14cbcSMatt Macy * the zthr_state_lock. That is because this function should 458eda14cbcSMatt Macy * only be called from the zthr_func to check if someone has 459eda14cbcSMatt Macy * issued a zthr_cancel() on the thread. If there is a zthr_cancel() 460eda14cbcSMatt Macy * happening concurrently, attempting to grab the request lock 461eda14cbcSMatt Macy * here would result in a deadlock. 462eda14cbcSMatt Macy * 463eda14cbcSMatt Macy * By grabbing only the zthr_state_lock this function is allowed 464eda14cbcSMatt Macy * to run concurrently with a zthr_cancel() request. 465eda14cbcSMatt Macy */ 466eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 467eda14cbcSMatt Macy boolean_t cancelled = t->zthr_cancel; 468eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 469eda14cbcSMatt Macy return (cancelled); 470eda14cbcSMatt Macy } 471eda14cbcSMatt Macy 472*e92ffd9bSMartin Matuska boolean_t 473*e92ffd9bSMartin Matuska zthr_iscurthread(zthr_t *t) 474*e92ffd9bSMartin Matuska { 475*e92ffd9bSMartin Matuska return (t->zthr_thread == curthread); 476*e92ffd9bSMartin Matuska } 477*e92ffd9bSMartin Matuska 478eda14cbcSMatt Macy /* 479eda14cbcSMatt Macy * Wait for the zthr to finish its current function. Similar to 480eda14cbcSMatt Macy * zthr_iscancelled, you can use zthr_has_waiters to have the zthr_func end 481eda14cbcSMatt Macy * early. Unlike zthr_cancel, the thread is not destroyed. If the zthr was 482eda14cbcSMatt Macy * sleeping or cancelled, return immediately. 483eda14cbcSMatt Macy */ 484eda14cbcSMatt Macy void 485eda14cbcSMatt Macy zthr_wait_cycle_done(zthr_t *t) 486eda14cbcSMatt Macy { 487eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 488eda14cbcSMatt Macy 489eda14cbcSMatt Macy /* 490eda14cbcSMatt Macy * Since we are holding the zthr_state_lock at this point 491eda14cbcSMatt Macy * we can find the state in one of the following 5 states: 492eda14cbcSMatt Macy * 493eda14cbcSMatt Macy * [1] The thread has already cancelled, therefore 494eda14cbcSMatt Macy * there is nothing for us to do. 495eda14cbcSMatt Macy * [2] The thread is sleeping so we set the flag, broadcast 496eda14cbcSMatt Macy * the CV and wait for it to exit. 497eda14cbcSMatt Macy * [3] The thread is doing work, in which case we just set 498eda14cbcSMatt Macy * the flag and wait for it to finish. 499eda14cbcSMatt Macy * [4] The thread was just created/resumed, in which case 500eda14cbcSMatt Macy * the behavior is similar to [3]. 501eda14cbcSMatt Macy * [5] The thread is the middle of being cancelled, which is 502eda14cbcSMatt Macy * similar to [3]. We'll wait for the cancel, which is 503eda14cbcSMatt Macy * waiting for the zthr func. 504eda14cbcSMatt Macy * 505eda14cbcSMatt Macy * Since requests are serialized, by the time that we get 506eda14cbcSMatt Macy * control back we expect that the zthr has completed it's 507eda14cbcSMatt Macy * zthr_func. 508eda14cbcSMatt Macy */ 509eda14cbcSMatt Macy if (t->zthr_thread != NULL) { 510eda14cbcSMatt Macy t->zthr_haswaiters = B_TRUE; 511eda14cbcSMatt Macy 512eda14cbcSMatt Macy /* broadcast in case the zthr is sleeping */ 513eda14cbcSMatt Macy cv_broadcast(&t->zthr_cv); 514eda14cbcSMatt Macy 515eda14cbcSMatt Macy while ((t->zthr_haswaiters) && (t->zthr_thread != NULL)) 516eda14cbcSMatt Macy cv_wait(&t->zthr_wait_cv, &t->zthr_state_lock); 517eda14cbcSMatt Macy 518eda14cbcSMatt Macy ASSERT(!t->zthr_haswaiters); 519eda14cbcSMatt Macy } 520eda14cbcSMatt Macy 521eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 522eda14cbcSMatt Macy } 523eda14cbcSMatt Macy 524eda14cbcSMatt Macy /* 525eda14cbcSMatt Macy * This function is intended to be used by the zthr itself 526eda14cbcSMatt Macy * to check if another thread is waiting on it to finish 527eda14cbcSMatt Macy * 528eda14cbcSMatt Macy * returns TRUE if we have been asked to finish. 529eda14cbcSMatt Macy * 530eda14cbcSMatt Macy * returns FALSE otherwise. 531eda14cbcSMatt Macy */ 532eda14cbcSMatt Macy boolean_t 533eda14cbcSMatt Macy zthr_has_waiters(zthr_t *t) 534eda14cbcSMatt Macy { 535eda14cbcSMatt Macy ASSERT3P(t->zthr_thread, ==, curthread); 536eda14cbcSMatt Macy 537eda14cbcSMatt Macy mutex_enter(&t->zthr_state_lock); 538eda14cbcSMatt Macy 539eda14cbcSMatt Macy /* 540eda14cbcSMatt Macy * Similarly to zthr_iscancelled(), we only grab the 541eda14cbcSMatt Macy * zthr_state_lock so that the zthr itself can use this 542eda14cbcSMatt Macy * to check for the request. 543eda14cbcSMatt Macy */ 544eda14cbcSMatt Macy boolean_t has_waiters = t->zthr_haswaiters; 545eda14cbcSMatt Macy mutex_exit(&t->zthr_state_lock); 546eda14cbcSMatt Macy return (has_waiters); 547eda14cbcSMatt Macy } 548