319 lines
9.6 KiB
Markdown
319 lines
9.6 KiB
Markdown
# Minetest asyncio
|
|
|
|
A lua coroutine wrapper thing. Inspired by Python's
|
|
[asyncio](https://docs.python.org/3/library/asyncio.html). As with Python's
|
|
asyncio, coroutines run in the main thread and must voluntarily give up
|
|
control.
|
|
|
|
*Note that this doesn't come with I/O-related functions, it is just called
|
|
`asyncio` to not conflict with the similar-but-different async mod.*
|
|
|
|
## Example code
|
|
|
|
### "Plain" lua
|
|
|
|
```lua
|
|
local function iter(a)
|
|
local stage = 0
|
|
return function()
|
|
if stage == 0 then
|
|
stage = 1
|
|
return 'Iteration started.'
|
|
elseif stage == 2 then
|
|
return
|
|
elseif a < 100 then
|
|
a = a * 2
|
|
return a
|
|
else
|
|
stage = 2
|
|
return 'Finished iterating.'
|
|
end
|
|
end
|
|
end
|
|
|
|
function itertest(a)
|
|
for i in iter(a) do
|
|
print('Iterator test: ' .. i)
|
|
end
|
|
end
|
|
|
|
print('Non-async code started, deferring...')
|
|
minetest.after(0, function()
|
|
print('Non-async code resumed, using iterator.')
|
|
itertest(1)
|
|
print('Finished iterating, waiting 1 second.')
|
|
minetest.after(1, function()
|
|
print('Waiting another two seconds.')
|
|
minetest.after(2, function()
|
|
print('Done!')
|
|
end)
|
|
end)
|
|
end)
|
|
```
|
|
|
|
### "Plain" lua syntax with asyncio
|
|
|
|
```lua
|
|
local iter = asyncio.coroutine(function(a)
|
|
asyncio.yield('Iteration started.')
|
|
while a < 100 do
|
|
a = a * 2
|
|
asyncio.yield(a)
|
|
end
|
|
asyncio.yield('Finished iterating.')
|
|
end)
|
|
|
|
itertest = asyncio.coroutine(function(a)
|
|
for i in asyncio.iter(iter, a) do
|
|
print('Iterator test: ' .. i)
|
|
end
|
|
end)
|
|
|
|
asyncio.run(function()
|
|
print('async code started, deferring...')
|
|
asyncio.defer()
|
|
print('async code resumed, using async iterator')
|
|
itertest(1)
|
|
print('Finished iterating, waiting 1 second.')
|
|
asyncio.sleep(1)
|
|
print('Waiting another two seconds without asyncio.sleep.')
|
|
asyncio.await(minetest.after(2, asyncio.resume))
|
|
print('Done!')
|
|
end)
|
|
```
|
|
|
|
### Entirely asyncio lua
|
|
|
|
```lua
|
|
local async function iter(a)
|
|
asyncio.yield('Iteration started.')
|
|
while a < 100 do
|
|
a = a * 2
|
|
asyncio.yield(a)
|
|
end
|
|
asyncio.yield('Finished iterating.')
|
|
end
|
|
|
|
async function itertest(a)
|
|
async for i in iter(a) do
|
|
print('Iterator test: ' .. i)
|
|
end
|
|
end
|
|
|
|
print('async code started, deferring...')
|
|
asyncio.defer()
|
|
print('async code resumed, using async iterator')
|
|
itertest(1)
|
|
print('Finished iterating, waiting 1 second.')
|
|
asyncio.sleep(1)
|
|
print('Waiting another two seconds without asyncio.sleep.')
|
|
asyncio.await(minetest.after(2, asyncio.resume))
|
|
print('Done!')
|
|
```
|
|
|
|
## How it works
|
|
|
|
This mod is a lightweight-ish wrapper around
|
|
[lua coroutines](https://www.lua.org/pil/9.1.html). The `async function` and
|
|
`async for` syntax can only be used with `asyncio.dofile` and is run through a
|
|
primitive transpiler(?) that converts the syntax into valid lua.
|
|
|
|
## API
|
|
|
|
The following functions are available both inside and outside of coroutines:
|
|
- `asyncio.run(function(...))`: Executes a function inside an asyncio
|
|
coroutine.
|
|
- `asyncio.coroutine(function(...))`: A function wrapper that runs the function
|
|
inside a coroutine.
|
|
- `asyncio.exec(...)`: Similar to `asyncio.loadstring(...)()`.
|
|
|
|
`asyncio.loadstring`, `asyncio.loadfile`, and `asyncio.dofile` work the same
|
|
way as their built-in counterparts, however these functions also accept
|
|
`async function` and `async for` syntax, and the code itself is executed inside
|
|
a coroutine.
|
|
|
|
The following functions only work inside coroutines:
|
|
- `asyncio.await(...)`: Suspends execution of the current coroutine until
|
|
`asyncio.resume` is called. If `asyncio.resume` is never called, the
|
|
coroutine will be garbage collected by lua.
|
|
Example: `asyncio.await(function_with_callback(params, asyncio.resume))`
|
|
- `asyncio.resume`: This should not be called inside the coroutine itself,
|
|
instead used as a callback function. `asyncio.resume` changes depending on
|
|
what coroutine is running, and is `nil` outside of coroutines.
|
|
- `asyncio.sleep(delay)`: Suspends execution of the current coroutine for
|
|
`delay` seconds. Internally, this just calls
|
|
`asyncio.await(minetest.after(delay, asyncio.resume))`.
|
|
- `asyncio.defer()`: Defers execution of this coroutine until the next
|
|
globalstep.
|
|
- `asyncio.optional_defer()`: Allows asyncio to defer, however leaves the
|
|
decision to actually defer up to asyncio. Currently, it will only call
|
|
`defer()` if it has been over 5ms since the previous "successful"
|
|
`optional_defer()`, however this may change in the future. Returns `true`
|
|
if `asyncio.defer()` was actually called, otherwise `false`.
|
|
- `asyncio.odefer()`: Alias for `asyncio.optional_defer`.
|
|
- `asyncio.iter(func, ...)`: Calls `func` as an asyncio iterator.
|
|
- `asyncio.pairs(obj, defer_every)`: Similar to `pairs()`, but calls
|
|
`asyncio.defer()` every `defer_every` iterations. Must be called in an
|
|
`async for` loop or with `asyncio.iter()`. If `defer_every` is unspecified,
|
|
`asyncio.optional_defer()` is called on every iteration instead.
|
|
- `asyncio.ipairs(obj, defer_every)`: The same as `asyncio.pairs`, but with
|
|
`ipairs`.
|
|
- `asyncio.yield(...)`: Used to yield values in asyncio iterators. This
|
|
function is slightly different from `coroutine.yield` and should not be
|
|
used interchangeably with it.
|
|
- `asyncio.running`: A variable, set to `true` when inside a coroutine. Do not
|
|
manually set this, or it will break.
|
|
- `asyncio.run_async(function(...))`: Runs `function(...)` in a separate
|
|
coroutine, so it does not block the current one.
|
|
- `asyncio.domodfile(file)`: Calls something similar to
|
|
`asyncio.dofile(current_modpath .. '/' .. file)`.
|
|
|
|
## Custom lua syntax
|
|
|
|
The custom lua syntax introduced by this mod is "transpiled" back into regular
|
|
lua, and can only be used with `asyncio.dofile` etc.
|
|
|
|
### Pre-runtime transpiling
|
|
|
|
**If you are going to be using Minetest's insecure environment from a file
|
|
using this custom syntax, I strongly recommend avoiding `asyncio.loadfile`
|
|
etc.**
|
|
|
|
To invoke the transpiler outside of Minetest, you can simply run
|
|
`lua transpile.lua /path/to/file.async.lua`. This should print the transpiled
|
|
code. If you want it to write the transpiled code to a different file, provided
|
|
you are on a UNIX-like OS, you can run
|
|
`lua transpile.lua file.async.lua > file.lua`.
|
|
|
|
*Note that `async for` loops will always be mangled ensure compatibility with
|
|
Lua 5.1.*
|
|
|
|
## Coroutines
|
|
|
|
asyncio coroutines can be created in one of two ways:
|
|
|
|
```lua
|
|
func1 = asyncio.coroutine(function(a, b, c))
|
|
return a + 1, b + 1, c + 1
|
|
end)
|
|
|
|
async function func2(a, b, c)
|
|
return a + 1, b + 1, c + 1
|
|
end
|
|
```
|
|
|
|
The `async function` sytnax will only work in lua code loaded with
|
|
`asyncio.dofile` etc.
|
|
|
|
The return value of coroutine functions will be lost if they are called from
|
|
other non-coroutine functions.
|
|
|
|
*You don't have to `await` coroutines from other coroutines, and calling
|
|
coroutines from non-asynchronous code will work correctly.*
|
|
|
|
## Iterators
|
|
|
|
asyncio iterators can only be used inside coroutines.
|
|
|
|
```lua
|
|
for value in asyncio.iter(iterable_function, parameters) do
|
|
print(value)
|
|
end
|
|
|
|
-- "async for" is recommended over asyncio.iter and should be used where
|
|
-- possible.
|
|
async for value in iterable_function(parameters) do
|
|
print(value)
|
|
end
|
|
```
|
|
|
|
Without LuaJIT, because of coroutine limitations in Lua 5.1, `asyncio.iter`
|
|
calculates and stores all iteration values before starting the loop. This
|
|
greatly reduces performance, especially when using `asyncio.pairs` or
|
|
`asyncio.ipairs`. `async for` does not have this limitation, however, and is
|
|
therefore recommended over `asyncio.iter`.
|
|
|
|
```lua
|
|
async for k, v in asyncio.pairs(minetest.registered_items) do
|
|
-- Some resource-intensive thing here.
|
|
end
|
|
```
|
|
|
|
*For another iterator example, see [here](#entirely-asyncio-lua).*
|
|
|
|
## HTTP(S) requests
|
|
|
|
As to not circumvent mod security, there are no asynchronous HTTP fetch
|
|
functions, and there are no helpers to create fetch functions.
|
|
|
|
To send an HTTP(S) request from an asyncio coroutine, you can call
|
|
`asyncio.await(http.fetch(req, asyncio.resume))`, where `http` is the name of
|
|
the [`HTTPApiTable`](https://github.com/minetest/minetest/blob/master/doc/lua_api.txt#L4746). This will return the response table.
|
|
|
|
Alternatively, if you are calling `http.fetch` multiple times, it may be easier
|
|
to define an asyncio HTTP fetch function:
|
|
|
|
```lua
|
|
local function http_fetch(req)
|
|
return asyncio.await(http.fetch(req, asyncio.resume))
|
|
end
|
|
```
|
|
|
|
## `wait until`
|
|
|
|
asyncio introduces a `wait until <condition>` statement, that checks
|
|
`condition` every globalstep until it evaluates to `true`, and then continues
|
|
executing the thread.
|
|
|
|
**Do not use `wait until` unless there is no other way, it can and will create
|
|
lag.**
|
|
|
|
## Locks and events
|
|
|
|
asyncio adds `asyncio.Lock`s and `asyncio.Event`s.
|
|
|
|
These are similar to
|
|
[Python's asyncio locks](https://docs.python.org/3/library/asyncio-sync.html).
|
|
|
|
Example:
|
|
|
|
```lua
|
|
local lock = asyncio.Lock()
|
|
async function do_something()
|
|
print('Acquiring lock...')
|
|
lock:acquire()
|
|
print('Acquired the lock, doing something...')
|
|
asyncio.sleep(5)
|
|
print('Releasing the lock...')
|
|
lock:release()
|
|
end
|
|
|
|
for i = 1, 5 do
|
|
asyncio.run_async(do_something)
|
|
end
|
|
|
|
-- Wait for the above functions to finish.
|
|
lock:acquire()
|
|
lock:release()
|
|
|
|
-- Events
|
|
local event = asyncio.Event()
|
|
async function do_something_else()
|
|
print('Waiting on the event...')
|
|
event:wait()
|
|
print('Event triggered, waiting 5 seconds.')
|
|
asyncio.sleep(5)
|
|
print('Done.')
|
|
end
|
|
|
|
for i = 1, 5 do
|
|
asyncio.run_async(do_something_else)
|
|
end
|
|
|
|
-- Set the event
|
|
asyncio.sleep(5)
|
|
print('Running event:set()...')
|
|
event:set()
|
|
print('Finished running event:set().')
|
|
```
|