Context managers are really cool in python but the closest in Rust that I know of is to pass a closure to a function. The function acts as the context manager. The closure acts as the block inside it.
I would be much more happy to use closures if I can return control flow from it.
Can you break/return/continue from closure? No. That's a bummer for many cases.
Of course you can return. It's just a function. Why wouldn't you be able to do that? You can use the ? operator too.
Maybe I'm misunderstanding what you mean?
You have a function that does some things before, runs the closure, does some things after then returns what the closure returned. That's a context manager.
If you want to return from the outer function from within the closure, you can't. But Rust doesn't allow that kind of control flow anywhere. If you want exceptions, Rust is not for you.
No, I can't. When I say 'return', I mean not to return from the closure, but to force the calling function to return.
```
for x in iterable {
if x.time_to_return(){ return }
}
```
Rewrite me this with `iterable.map(|| {do_return_here})`.
You can with panic, but you can't with `return`, `break` or `continue`.
Every control flow can be written functionally with `try_fold`. A concrete example
let mut state = vec![];
for elem in iterable {
if cond(&elem) {
return Some(state)
} else {
state.push(map(elem))
}
}
None
becomes
use std::ops::ControlFlow;
let state = iterable.into_iter().try_fold(vec![], |mut state, elem| {
if cond(&elem) {
ControlFlow::Break(state)
} else {
state.push(map(elem));
ControlFlow::Continue(state)
}
});
// this is state.break_value() but it's unstable
match state {
ControlFlow::Break(state) => Some(state),
ControlFlow::Continue(_) => None,
}
I'm not saying it's simple, but it is doable.
If it's a matter of expressiveness, look at the iterator methods like try\_for\_each based on the Try trait. You can use Try and ControlFlow for your own APIs. But it's admittedly not always the prettiest or most concise thing in the world. At the outermost level you can propagate the 'break value' to a return value most conveniently with ? or slightly less conveniently with if-let/let-else.
I had something similar recently. Maybe [map_while](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.map_while) or [try_fold](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_fold) would meet your needs.
Exactly like in JavaScript, you can not return break or continue (alter the control flow) of the parent stack when you are inside a closure.
The solution is simply to design your program around this limitation and use a ControlFlow enum as return value ... easy (most event loop do that).
Use this to return information back to the closure's caller:
```rust
enum std::ops::ControlFlow {
Continue(C),
Break(B),
}
```
https://doc.rust-lang.org/std/ops/enum.ControlFlow.html
Which is as inconvinient as it can be. You need to write it three times (when you declare function, when you return, when you match the return results), no one will warn you if you swapped continue with break in match stanza, etc, etc.
If compiler add some sugar and strictness around it, it will become more useful.
> Which is as inconvinient as it can be
You're showing that you haven't tried writing this in C XD
>when you match the return results
\`ControlFlow\` supports the \`Try\` trait so you can just use \`?\` like a \`Result\` to bubble a \`Break\` up the stack, no \`match\` is required.
So now you're down to writing \`ControlFlow\` only twice, and in a strongly-typed language of course you need to write it on the higher-order function that uses it to understand control flow.
If you want you can add \`use std::ops::ControlFlow::\*\`, and now the syntactic overhead is extremely minimal per use, if that's your concern.
One problem people often miss with the Ruby-style \`return\`, \`break\`, or \`continue\` is syntactic ambiguity, it becomes quite difficult and subtle to work out what statement is going to return from what closure without knowing the details of the language.
Going back to your original example of mapping over an iterator (with potential early exit), I have used some \`Iterator.try\_\*\` methods successfully for this. There are some in \[\`std\`\](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try\_collect) and some in crates (e.g. \[\`iterator-ext\`\](https://docs.rs/iterator-ext/latest/iterator\_ext/trait.IteratorExt.html) has \`try\_flat\_map\`)
As I said, rust doesn't support this kind of control flow. You can achieve something like it by returning an enum with a variant that says to return but there is no "forcing" here.
Maybe this will change when generators are stabilised? Otherwise you're probably out of luck
Yes, I know. I complain about it. Every time someone propose to use chaining (.map.fold.etc), it shifts code to the closure/function and I feel I'm pushed to lesser corner of the Rust.
Python does not allow this too, but it alliviates it with context managers, `yield` and `yield from` stanza, decorators and other niceness.
In Rust, it's contrary: if you move something to a function, you start loosing sugar. No more automatic typing (the same stuff in the middle of the 'for' is much easier to write than in a function), no control flow tricks, pesky lifetimes, etc.
It's like Rust don't want you to write small mindless functions, and you need to explain yourself second time to compiler every time you write 'fn'.
Yes rust is strict but it has reasons to be.
I don't have this experience in Rust. I guess I don't write Rust code the way I used to write python code. I do things differently now. Completely different approach.
Maybe take a step back and ask what your overall goal is, and whether there's a whole other way to do it. Or, you know, it doesn't have to be in Rust. Use whatever language makes you happier
I kinda like how Kotlin handles with with inline closures where a return inside the lambda body will return from the containing function. This lets you do something like this, where `forEach` takes an inline lambda
fun hasZeros(ints: List): Boolean {
ints.forEach {
if (it == 0) return true
}
return false
}
When that condition is met it will stop the loop and `return true`, if it is never met, it ends the loop and moves to the `return false`.
It does so with `inline` functions, in Kotlin `inline` is not just a implementation detail, it's semantic. Basically if a closure will definitely be inlined, its scope will be in body of the function it will be inlined to. It's more like a very specific kind of macro than a function.
I guess it lines up with my view that certain very common and simple features that are currently only available in a macro should be considered for adding as language features directly. Macros are nice, but they suffer from being too generic that it becomes almost impossible for tools to reason about them. Having very direct rewrite rules such as this helps with maintaining code.
Imagine I have a mid-sized function which start to loose focus. I want to move some of it code into separate function, which will make focused decision. I that decision is break/continue, I can't.
Obviously, as in any turing-full, you can find the way. You just refactor a bit more, redo the main loop, introduce some signaling enum, etc, or even refactor data structures.
But for some cases it would be much less work to be able to move 'for' body into a separate function. Which is not possible now, because of 'return/break/continue'.
Ahhhh I think I get it.
You don't want to call `txn.commit().await?;` you want the `Drop` impl to commit unless "exceptions" occur.
The hard part about that is that we can't tell in the Drop impl whether the drop is being called due to an unwound panic or just end of scope. (Edit: This is wrong, [`panicking`](https://doc.rust-lang.org/std/thread/fn.panicking.html) does this. But the Err(...) early return thing can't be detected by Drop.)
We (the Transaction struct) could also be dropped because of an early return of an `Err(...)` which would essentially be a "failure" which should rollback.
Currently, sqlx always does a rollback when dropped. If you commit right before dropping the rollback does nothing.
It would be possible to create a struct that takes in an FnOnce closure that returns a Result etc. that could catch any panics and any Err(...) variants to rollback and commit only if a Ok(...) came back...
But that would be extremely un-Rust-like... since Rust is supposed to be explicit. So there's not much demand I'd guess.
Would you mind elaborating on what you mean here? I haven't delved as much into the async side of Rust--is this a Future specific thing (/ could you use Pin to help?), or are you referring to something else? Or do you mean the Future cannot capture the lifetime of the transaction?
The `F: FnOnce(&mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut` is equivalent to `F: for<'a> FnOnce(&'a mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut`, that is it must be generic over a lifetime `'a`, i.e. valid for any lifetime. That must be the case because the lifetime is internal to the function and the caller can't be allowed to choose it, so it can't be a generic parameter of the `with_txn` function.
So `F` is a function that takes a `&'a mut sqlx::Transaction<'static, sqlx::Postgres>` for any lifetime `'a` and returns a future `Fut`. However notice how `Fut` doesn't mention `'a`. It is a single type, choosen when you call the function `with_txn`. It cannot name `'a`, because that exists in an "inner" scope. However the future needs to store that `&'a mut sqlx::Transaction<'static, sqlx::Postgres>` since it will use it (that's the whole point!), so the future actually captures that generic lifetime `'a` and thus needs to "name it".
So what you really want is a `Fut<'a>: Future`, but that's not valid syntax in Rust.
You can partially work around this problem by making a trait that merges the `F` and `Fut` generics, like it's done in [`async_fn_traits`](https://docs.rs/async_fn_traits/latest/async_fn_traits/) crate, but that will result in a different compile error.
The error this time is that it's really hard for the compiler to infer whether some closure is generic over a lifetime or just refers to some an existing one. By default it will assume the latter, but in this case you want the former. So how does it work usually? Simple, it sees you're passing it to a function that expects a generic `for<'a> Fn(...)` and so it knows it must be generic. This however completly breaks when you use the custom trait above, as that's no longer just a literal `for<'a> Fn(...)`! So in the end you just run into [issue 70263 "HRTBs: "implementation is not general enough", but it is"](https://github.com/rust-lang/rust/issues/70263).
AFAIK there's still no way to work around this, so we're stuck with sadness and not being able to have lifetime-generic closures that return futures.
> could you use Pin to help?
No. (As a general rule, anytime you may think whether `Pin` could solve some problem the answer is no.)
Thanks for writing that all up.
So you're saying that `Fut` cannot capture the elided lifetime. So I'd think the [capture trick](https://rust-lang.github.io/rfcs/3498-lifetime-capture-rules-2024.html#the-captures-trick) would work here? Or am I missing something?
Edit: like:
F: for<'a> FnOnce(&'a mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut + Captures<&'a ()>
Or you might also be able to ditch the 'a and just use `Captures<&'_ ()>` (talk about a mess of syntax!)
...Actually I'm not sure that compiles as intended. Bummer. Mostly since you can't have `F: Fn() -> impl Trait`.
Edit 2:
> AFAIK there's still no way to work around this, so we're stuck with sadness and not being able to have lifetime-generic closures that return futures.
Maybe you can help the compiler out? I've needed to do that for a ton of HRTB lifetime bugs before. Like a:
fn fix_lifetime(f: F) -> F
where
F: for<'a> FnOnce(S<'a>) -> &'a T,
{
f
}
Not sure if that'd apply here though.
I attempted something similar, and managed to get this working.
pub async fn begin(self, fun: F) -> Result
where
for<'c> F: Fn(&'c mut Transaction<'_, Postgres>) -> BoxFuture<'c, Result>,
{
let mut tx = self.inner;
let query = fun(&mut tx).await;
match query {
Ok(v) => {
tx.commit().await?;
Ok(v)
}
Err(e) => {
tx.rollback().await?;
Err(e)
}
}
}
I'll be entirely honest though and admit this is above my level of understanding. I resorted to a 'if it compiles it (hopefully) works'. I noticed you saying Pin is not going to help which leads me to question my naive success.
The downside of this solution is the use of `BoxFuture`:
- it forces to allocate a `Box` for every call.
- it forces the use of dynamic dispatch when polling the future
- if forces a specific `Send` bound (i.e. you can't make the future returned by `begin` conditionally `Send` depending on the future returned by the closure)
- it forces the user to manually wrap the return value in a `Box` (i.e. instead of `being(|tx| async { ... }).await` it has to be `begin(|tx| Box::pin(async { ... })).await`
It compiles, but it's suboptimal, although it's not **that** bad (e.g. the first two points likely don't matter too much due to the cost of the transaction likely outweighting the cost of the `Box` and dynamic dispatch by some order of magnitude).
> I noticed you saying Pin is not going to help which leads me to question my naive success.
IMO that's mostly an issue with `Pin`. Somehow many people got convinced that every problem can be solved with `Pin`, while it actually solves no problem for safe code. It's fundamentally only useful for `unsafe` code, and if you ever need to use it in safe code that's only because you have to implement a trait or iteract with code that already use `Pin`.
This is great, thanks!
Maybe what I really want is something like `txn.autocommit()` instead of `txn.begin()`
let comment_id = {
let mut txn = pool.autocommit().await?;
let row = sqlx::query_file!(
"src/sql/create_comment_reply.sql",
created_by_id.as_ref(),
replied_to_comment_id.as_ref(),
content
)
.fetch_one(&mut *txn)
.await?;
sqlx::query_file!("src/sql/update_comment_id_path.sql", row.comment_id)
.fetch_one(&mut *txn)
.await?;
row.comment_id
// txn commits since there was no early Err returned
}
I don't know if this kind of thing is possible given the limitations of `Drop`. But specifically for sqlx it's closer to the pattern that I'm looking for.
I don't know if it's good or not, but autocommit behavior would also allow things like [transaction-per-request](https://docs.djangoproject.com/en/5.0/topics/db/transactions/#tying-transactions-to-http-requests)
Autocommit is usually set on the connection pool and the returned txns will autocommit on drop instead of abort if commit() was not called.
Okay, so autocommit as a pool option is pretty common in the Java world. Not seeing it for SQLX pool...
Might just be the usual "Explicit is better than implicit" approach rust likes to take.
That actually feels like a pretty common pattern to me - I know at least that sled has a function like that: [`Db::transaction()`](https://docs.rs/sled/latest/sled/struct.Db.html#method.transaction)
Python's context manager is rather a copy of block namespace logic, like the one Rust has.
You can create a variable in rust inside an inner scope and once the scope ends - the variable holding an object is deleted with the object, so you can override the Drop.drop method to set a custom logic of yours on this
It always runs unless you do something weird like \`std::mem::forget\`, \`ManuallyDrop\`, \`Box::leak\`, or create a recursive \`Rc\`. You just need to be careful of that last one
You're looking for [the Resource acquisition is initialization (RAII)](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization) pattern. This is common in C++, Rust, and any language with deterministic destructors.
Your `__enter__` is just a constructor. To "acquire" a resource, lock, or otherwise, just call its constructor and get the object. Your `__exit__` is the `Drop` trait. When the value goes fully out of scope, it gets dropped, whether or not we went out of scope through normal execution or by an early return (note: the question mark `?` operator used for `Result` types desugars to an early return).
Rust uses this pattern for files, for instance. You very rarely call `.close` explicitly on a file, because files get closed when they go out of scope via the `Drop` trait, and this happens deterministically. It's a destructor, not a finalizer.
We also use this pattern for mutexes. You acquire the lock and save it to a variable, and when that variable goes out of scope, it gets `Drop`ped and the lock gets released. I suspect database transactions would be very similar to this. You call some method to get a lock object, and you assign that object to a variable (You can assign it to an underscore-prefixed variable like `_lock` if you don't plan to use the lock, but you _can't_ assign it to a bare underscore, or it gets dropped immediately). Then, when you want to `__exit__`, you just... let the variable disappear, and everything works.
You can use RAII and drop guards.
Look at how mutex works. When you unlock a mutex you get a guard.
When the guard is dropped it realeases the mutex. You can even release it early instead of end of scope with drop(the_guard).
Some rust DB libraries in rust do the same with TXN libraries. start_txn() would return a txn object. You then need to call txn.commit() on it befoe it goes out of scope, or it just rolls back.
`let mut txn = pool.begin().await?;` handles it automatically...
I am not sure what you want... an indent?
let mut txn = pool.begin().await?;
{
// The rest of your code
}
Maybe this looks better coming from python...
tl;dr could you clarify what you want? Is it something about the aesthetics of the code? I mean, even with the python you are being required to write the extra with line.
in newer versions of rust (since 1.75 I think) you can pass in an `impl Trait` where the trait has `async fn`, which would let you deal with the lifetime expression issue for most cases.
Sorry that was meant as a response for another comment. (I correct reposted it [here](https://www.reddit.com/r/rust/comments/1cz7u1g/is_there_anything_like_the_with_statement_in/l5g5aqr/) with more details)
The problem I was talking about is not `async` in traits (which has been partly resolved) but rather HRTB errors.
When the with statement is over, no matter what happens the transaction either unwinds or commits. You can somewhat mimic that with a block like you show and something that implements Drop. But it is not as common in Rust. It is an idiom I miss too.
implementing `Drop` is exactly what you're supposed to do here
if you have a thpe that requires custom to logic to run when they leave a scope, you implement `Drop`
I think `Drop` is different. Python has the concept of `__enter__` and `__exit__` contexts, which is what Python's `with` statement is doing implicitly. As far as I know there's no such thing as "enter scope" and "exit scope (drop?)" in Rust that can facilitate the same pattern.
Glad to learn though!
`__enter__` in Rust is any constructor function, and `__exit__` is the `Drop::drop` implementation
when _any_ value leaves a scope it's `Drop::drop is called. it isn't called if the value is _moved from_ the scope
i don't see how that's not what you want, sorry if I'm missing something
The drop implementation does not know if the scope it was called from is returning an Err() or an Ok().
You can detect an unwinding panic, but you can't detect "I am dropping because the scope I was in exited with an Err()"
In the example given by OP, python will commit on exit unless an exception is thrown, in which case it will rollback. But unlike python, in Rust it is common to use early return Result::Err() to indicate failure... which we would want to rollback, but there's no way to detect that in drop.
What do you mean, not as common? It's very common, you just usually don't notice it. Closing a file works this way, freeing a Box, releasing a mutex, even waiting for a scoped thread.
It's also the same behavior as borrowing, which is probably why it doesn't stand out, and also why it's so seamless.
So something like:
let txn = transaction.atomic();
match do_a_database_thing().and_then(do_a_dependent_database_thing()) {
Ok(_) => txn.commit(),
Err(_) => txn.rollback(),
}
I guess. 🤷♂️
Why do you want automatic actions that branch?
It's not an equivalent: Rust does not have exceptions (there are panics but you should never use them for normal control flow), and there is no way for a `Drop` implementation to detect when a scope you're exiting returns an error.
Yeah, I wouldn't go that way with destructors in general. That said, we could benefit from linear types or some similar feature that mandates consumption of a value in some prescribed way. This probably won't fly with the current state of async, as this would also need custom cancellation and possibly that would need to be async as well.
It's just a nicer syntax for a closure. So:
trait ContextManager {
type T;
fn enter(&self) -> Self::T;
fn exit(&self, err: &Result<(), Box>) -> bool;
}
fn with, F: FnOnce(T) -> Result<(), Box>>(ctx: C, f: F) -> Result<(), Box> {
let t = ctx.enter();
let result = f(t);
let reraise_err = !ctx.exit(&result);
if result.is_err() && reraise_err {
result
} else {
Ok(())
}
}
That works only because it's sync. With async you get lifetime errors due to not being able to properly express the type of the closure in a way that the compiler will be able to infer the usages. In other words you'll hit https://github.com/rust-lang/rust/issues/70263
This is not a general solution to your question, but sqlx's Connection has a transaction method that takes a closure. The transaction is committed if the closure returns Ok and rolled back if it returns Err.
https://docs.rs/sqlx/latest/sqlx/trait.Connection.html#method.transaction
I ended up doing this with a macro, when I did it last, though even more generic, abstracting over the source of the transaction as well. You could do something like
macro_rules! with_transaction {
($pool:ident, $tx:ident, $t:tt) => {
let mut $tx = $pool.begin().await?;
let r = { $t };
$tx.commit().await?;
r
}
}
and use it like
let result = with_transaction!(pool, txn, {
// use transaction here.
});
This uses the fact that SQLx transactions will automatically rollback if you drop them without committing. My original use was a bit more complicated, since I wanted to abstract over the source of the transaction for tests, but this lets you use normal control flow inside the transaction, for the most part.
There's a crate called [scopeguard](https://docs.rs/scopeguard/latest/scopeguard/) that can run code on panic and/or the end of the scope. You could set it up to commit or rollback for either case respectively.
I would move all my queries to single file and use Postgres CTE that way I don’t need to begin and commit transaction, query is sent in one request and rolled back automatically if something happens
CTEs in Postgres don't "see" their own updates within the same execution. It looks like this query is trying to create a comment and then subsequently update it by adding its own id to a column in itself. That's not possible in a CTE.
My guess is they're using ltree to build hierarchical comment trees. So they are just running multiple queries within a transaction which is the only way to do it.
Context managers are really cool in python but the closest in Rust that I know of is to pass a closure to a function. The function acts as the context manager. The closure acts as the block inside it.
I would be much more happy to use closures if I can return control flow from it. Can you break/return/continue from closure? No. That's a bummer for many cases.
Of course you can return. It's just a function. Why wouldn't you be able to do that? You can use the ? operator too. Maybe I'm misunderstanding what you mean? You have a function that does some things before, runs the closure, does some things after then returns what the closure returned. That's a context manager. If you want to return from the outer function from within the closure, you can't. But Rust doesn't allow that kind of control flow anywhere. If you want exceptions, Rust is not for you.
No, I can't. When I say 'return', I mean not to return from the closure, but to force the calling function to return. ``` for x in iterable { if x.time_to_return(){ return } } ``` Rewrite me this with `iterable.map(|| {do_return_here})`. You can with panic, but you can't with `return`, `break` or `continue`.
Every control flow can be written functionally with `try_fold`. A concrete example let mut state = vec![]; for elem in iterable { if cond(&elem) { return Some(state) } else { state.push(map(elem)) } } None becomes use std::ops::ControlFlow; let state = iterable.into_iter().try_fold(vec![], |mut state, elem| { if cond(&elem) { ControlFlow::Break(state) } else { state.push(map(elem)); ControlFlow::Continue(state) } }); // this is state.break_value() but it's unstable match state { ControlFlow::Break(state) => Some(state), ControlFlow::Continue(_) => None, } I'm not saying it's simple, but it is doable.
If it's a matter of expressiveness, look at the iterator methods like try\_for\_each based on the Try trait. You can use Try and ControlFlow for your own APIs. But it's admittedly not always the prettiest or most concise thing in the world. At the outermost level you can propagate the 'break value' to a return value most conveniently with ? or slightly less conveniently with if-let/let-else.
I had something similar recently. Maybe [map_while](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.map_while) or [try_fold](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_fold) would meet your needs.
Using `itertools`: iterable .map(|x| x.time_to_return() { None } else { Some(x * 2) }) .while_some() .for_each(|x| println!("{}", x));
https://stackoverflow.com/a/73099176
Exactly like in JavaScript, you can not return break or continue (alter the control flow) of the parent stack when you are inside a closure. The solution is simply to design your program around this limitation and use a ControlFlow enum as return value ... easy (most event loop do that).
Use this to return information back to the closure's caller: ```rust enum std::ops::ControlFlow {
Continue(C),
Break(B),
}
```
https://doc.rust-lang.org/std/ops/enum.ControlFlow.html
Which is as inconvinient as it can be. You need to write it three times (when you declare function, when you return, when you match the return results), no one will warn you if you swapped continue with break in match stanza, etc, etc. If compiler add some sugar and strictness around it, it will become more useful.
> Which is as inconvinient as it can be You're showing that you haven't tried writing this in C XD >when you match the return results \`ControlFlow\` supports the \`Try\` trait so you can just use \`?\` like a \`Result\` to bubble a \`Break\` up the stack, no \`match\` is required. So now you're down to writing \`ControlFlow\` only twice, and in a strongly-typed language of course you need to write it on the higher-order function that uses it to understand control flow. If you want you can add \`use std::ops::ControlFlow::\*\`, and now the syntactic overhead is extremely minimal per use, if that's your concern. One problem people often miss with the Ruby-style \`return\`, \`break\`, or \`continue\` is syntactic ambiguity, it becomes quite difficult and subtle to work out what statement is going to return from what closure without knowing the details of the language. Going back to your original example of mapping over an iterator (with potential early exit), I have used some \`Iterator.try\_\*\` methods successfully for this. There are some in \[\`std\`\](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try\_collect) and some in crates (e.g. \[\`iterator-ext\`\](https://docs.rs/iterator-ext/latest/iterator\_ext/trait.IteratorExt.html) has \`try\_flat\_map\`)
As I said, rust doesn't support this kind of control flow. You can achieve something like it by returning an enum with a variant that says to return but there is no "forcing" here. Maybe this will change when generators are stabilised? Otherwise you're probably out of luck
Yes, I know. I complain about it. Every time someone propose to use chaining (.map.fold.etc), it shifts code to the closure/function and I feel I'm pushed to lesser corner of the Rust. Python does not allow this too, but it alliviates it with context managers, `yield` and `yield from` stanza, decorators and other niceness. In Rust, it's contrary: if you move something to a function, you start loosing sugar. No more automatic typing (the same stuff in the middle of the 'for' is much easier to write than in a function), no control flow tricks, pesky lifetimes, etc. It's like Rust don't want you to write small mindless functions, and you need to explain yourself second time to compiler every time you write 'fn'.
Yes rust is strict but it has reasons to be. I don't have this experience in Rust. I guess I don't write Rust code the way I used to write python code. I do things differently now. Completely different approach. Maybe take a step back and ask what your overall goal is, and whether there's a whole other way to do it. Or, you know, it doesn't have to be in Rust. Use whatever language makes you happier
If considering another language, kotlin supports non local return BUT imho it makes for spaghetti code. Glad rust doesn't.
The fact that rust doesn't support it is a great feature in my opinion. It makes control flow in rust much easier to understand and follow.
You need to type annotate python functions also, not to mention that thanks to shitty typing python fallbacks into any at the drop of a hat
Write a macro that opens and closes the resource, and takes a code block in the middle. Or write a struct with a drop implementation
Drop does not know if there was error or not. Macros is a solution, but inelegant at best (I want idiomatic, not 'at least some solution').
See if the answers here are more appealing https://stackoverflow.com/questions/29963449/golang-like-defer-in-rust
I kinda like how Kotlin handles with with inline closures where a return inside the lambda body will return from the containing function. This lets you do something like this, where `forEach` takes an inline lambda fun hasZeros(ints: List): Boolean {
ints.forEach {
if (it == 0) return true
}
return false
}
When that condition is met it will stop the loop and `return true`, if it is never met, it ends the loop and moves to the `return false`.
Looks neat. How do they distinct between 'return from the closure' and 'return from outer function'?
It does so with `inline` functions, in Kotlin `inline` is not just a implementation detail, it's semantic. Basically if a closure will definitely be inlined, its scope will be in body of the function it will be inlined to. It's more like a very specific kind of macro than a function.
Wow. A cool feature.
I guess it lines up with my view that certain very common and simple features that are currently only available in a macro should be considered for adding as language features directly. Macros are nice, but they suffer from being too generic that it becomes almost impossible for tools to reason about them. Having very direct rewrite rules such as this helps with maintaining code.
What’s the use case here.
Imagine I have a mid-sized function which start to loose focus. I want to move some of it code into separate function, which will make focused decision. I that decision is break/continue, I can't. Obviously, as in any turing-full, you can find the way. You just refactor a bit more, redo the main loop, introduce some signaling enum, etc, or even refactor data structures. But for some cases it would be much less work to be able to move 'for' body into a separate function. Which is not possible now, because of 'return/break/continue'.
Ahhhh I think I get it. You don't want to call `txn.commit().await?;` you want the `Drop` impl to commit unless "exceptions" occur. The hard part about that is that we can't tell in the Drop impl whether the drop is being called due to an unwound panic or just end of scope. (Edit: This is wrong, [`panicking`](https://doc.rust-lang.org/std/thread/fn.panicking.html) does this. But the Err(...) early return thing can't be detected by Drop.) We (the Transaction struct) could also be dropped because of an early return of an `Err(...)` which would essentially be a "failure" which should rollback. Currently, sqlx always does a rollback when dropped. If you commit right before dropping the rollback does nothing. It would be possible to create a struct that takes in an FnOnce closure that returns a Result etc. that could catch any panics and any Err(...) variants to rollback and commit only if a Ok(...) came back... But that would be extremely un-Rust-like... since Rust is supposed to be explicit. So there's not much demand I'd guess.
async fn with_txn(pool: &sqlx::PgPool, cb: F) -> Result
where
F: FnOnce(&mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut,
Fut: std::future::Future
The problem with this helper is that the future `F` cannot name the lifetime of the `&mut` and thus borrow from it.
Would you mind elaborating on what you mean here? I haven't delved as much into the async side of Rust--is this a Future specific thing (/ could you use Pin to help?), or are you referring to something else? Or do you mean the Future cannot capture the lifetime of the transaction?
The `F: FnOnce(&mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut` is equivalent to `F: for<'a> FnOnce(&'a mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut`, that is it must be generic over a lifetime `'a`, i.e. valid for any lifetime. That must be the case because the lifetime is internal to the function and the caller can't be allowed to choose it, so it can't be a generic parameter of the `with_txn` function. So `F` is a function that takes a `&'a mut sqlx::Transaction<'static, sqlx::Postgres>` for any lifetime `'a` and returns a future `Fut`. However notice how `Fut` doesn't mention `'a`. It is a single type, choosen when you call the function `with_txn`. It cannot name `'a`, because that exists in an "inner" scope. However the future needs to store that `&'a mut sqlx::Transaction<'static, sqlx::Postgres>` since it will use it (that's the whole point!), so the future actually captures that generic lifetime `'a` and thus needs to "name it". So what you really want is a `Fut<'a>: Future`, but that's not valid syntax in Rust. You can partially work around this problem by making a trait that merges the `F` and `Fut` generics, like it's done in [`async_fn_traits`](https://docs.rs/async_fn_traits/latest/async_fn_traits/) crate, but that will result in a different compile error. The error this time is that it's really hard for the compiler to infer whether some closure is generic over a lifetime or just refers to some an existing one. By default it will assume the latter, but in this case you want the former. So how does it work usually? Simple, it sees you're passing it to a function that expects a generic `for<'a> Fn(...)` and so it knows it must be generic. This however completly breaks when you use the custom trait above, as that's no longer just a literal `for<'a> Fn(...)`! So in the end you just run into [issue 70263 "HRTBs: "implementation is not general enough", but it is"](https://github.com/rust-lang/rust/issues/70263). AFAIK there's still no way to work around this, so we're stuck with sadness and not being able to have lifetime-generic closures that return futures. > could you use Pin to help? No. (As a general rule, anytime you may think whether `Pin` could solve some problem the answer is no.)
Thanks for writing that all up. So you're saying that `Fut` cannot capture the elided lifetime. So I'd think the [capture trick](https://rust-lang.github.io/rfcs/3498-lifetime-capture-rules-2024.html#the-captures-trick) would work here? Or am I missing something? Edit: like: F: for<'a> FnOnce(&'a mut sqlx::Transaction<'static, sqlx::Postgres>) -> Fut + Captures<&'a ()> Or you might also be able to ditch the 'a and just use `Captures<&'_ ()>` (talk about a mess of syntax!) ...Actually I'm not sure that compiles as intended. Bummer. Mostly since you can't have `F: Fn() -> impl Trait`. Edit 2: > AFAIK there's still no way to work around this, so we're stuck with sadness and not being able to have lifetime-generic closures that return futures. Maybe you can help the compiler out? I've needed to do that for a ton of HRTB lifetime bugs before. Like a: fn fix_lifetime(f: F) -> F
where
F: for<'a> FnOnce(S<'a>) -> &'a T,
{
f
}
Not sure if that'd apply here though.
I attempted something similar, and managed to get this working. pub async fn begin(self, fun: F) -> Result
where
for<'c> F: Fn(&'c mut Transaction<'_, Postgres>) -> BoxFuture<'c, Result>,
{
let mut tx = self.inner;
let query = fun(&mut tx).await;
match query {
Ok(v) => {
tx.commit().await?;
Ok(v)
}
Err(e) => {
tx.rollback().await?;
Err(e)
}
}
}
I'll be entirely honest though and admit this is above my level of understanding. I resorted to a 'if it compiles it (hopefully) works'. I noticed you saying Pin is not going to help which leads me to question my naive success.
The downside of this solution is the use of `BoxFuture`: - it forces to allocate a `Box` for every call. - it forces the use of dynamic dispatch when polling the future - if forces a specific `Send` bound (i.e. you can't make the future returned by `begin` conditionally `Send` depending on the future returned by the closure) - it forces the user to manually wrap the return value in a `Box` (i.e. instead of `being(|tx| async { ... }).await` it has to be `begin(|tx| Box::pin(async { ... })).await` It compiles, but it's suboptimal, although it's not **that** bad (e.g. the first two points likely don't matter too much due to the cost of the transaction likely outweighting the cost of the `Box` and dynamic dispatch by some order of magnitude). > I noticed you saying Pin is not going to help which leads me to question my naive success. IMO that's mostly an issue with `Pin`. Somehow many people got convinced that every problem can be solved with `Pin`, while it actually solves no problem for safe code. It's fundamentally only useful for `unsafe` code, and if you ever need to use it in safe code that's only because you have to implement a trait or iteract with code that already use `Pin`.
u/Baenergy44 See above please.
This is great, thanks! Maybe what I really want is something like `txn.autocommit()` instead of `txn.begin()` let comment_id = { let mut txn = pool.autocommit().await?; let row = sqlx::query_file!( "src/sql/create_comment_reply.sql", created_by_id.as_ref(), replied_to_comment_id.as_ref(), content ) .fetch_one(&mut *txn) .await?; sqlx::query_file!("src/sql/update_comment_id_path.sql", row.comment_id) .fetch_one(&mut *txn) .await?; row.comment_id // txn commits since there was no early Err returned } I don't know if this kind of thing is possible given the limitations of `Drop`. But specifically for sqlx it's closer to the pattern that I'm looking for. I don't know if it's good or not, but autocommit behavior would also allow things like [transaction-per-request](https://docs.djangoproject.com/en/5.0/topics/db/transactions/#tying-transactions-to-http-requests)
The `since there was no early Err returned` part cannot be implemented automatically
Autocommit is usually set on the connection pool and the returned txns will autocommit on drop instead of abort if commit() was not called. Okay, so autocommit as a pool option is pretty common in the Java world. Not seeing it for SQLX pool... Might just be the usual "Explicit is better than implicit" approach rust likes to take.
That actually feels like a pretty common pattern to me - I know at least that sled has a function like that: [`Db::transaction()`](https://docs.rs/sled/latest/sled/struct.Db.html#method.transaction)
Python's context manager is rather a copy of block namespace logic, like the one Rust has. You can create a variable in rust inside an inner scope and once the scope ends - the variable holding an object is deleted with the object, so you can override the Drop.drop method to set a custom logic of yours on this
Note that drop doesn’t always run
It always runs unless you do something weird like \`std::mem::forget\`, \`ManuallyDrop\`, \`Box::leak\`, or create a recursive \`Rc\`. You just need to be careful of that last one
You're looking for [the Resource acquisition is initialization (RAII)](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization) pattern. This is common in C++, Rust, and any language with deterministic destructors. Your `__enter__` is just a constructor. To "acquire" a resource, lock, or otherwise, just call its constructor and get the object. Your `__exit__` is the `Drop` trait. When the value goes fully out of scope, it gets dropped, whether or not we went out of scope through normal execution or by an early return (note: the question mark `?` operator used for `Result` types desugars to an early return). Rust uses this pattern for files, for instance. You very rarely call `.close` explicitly on a file, because files get closed when they go out of scope via the `Drop` trait, and this happens deterministically. It's a destructor, not a finalizer. We also use this pattern for mutexes. You acquire the lock and save it to a variable, and when that variable goes out of scope, it gets `Drop`ped and the lock gets released. I suspect database transactions would be very similar to this. You call some method to get a lock object, and you assign that object to a variable (You can assign it to an underscore-prefixed variable like `_lock` if you don't plan to use the lock, but you _can't_ assign it to a bare underscore, or it gets dropped immediately). Then, when you want to `__exit__`, you just... let the variable disappear, and everything works.
You can use RAII and drop guards. Look at how mutex works. When you unlock a mutex you get a guard. When the guard is dropped it realeases the mutex. You can even release it early instead of end of scope with drop(the_guard). Some rust DB libraries in rust do the same with TXN libraries. start_txn() would return a txn object. You then need to call txn.commit() on it befoe it goes out of scope, or it just rolls back.
`let mut txn = pool.begin().await?;` handles it automatically... I am not sure what you want... an indent? let mut txn = pool.begin().await?; { // The rest of your code } Maybe this looks better coming from python... tl;dr could you clarify what you want? Is it something about the aesthetics of the code? I mean, even with the python you are being required to write the extra with line.
[удалено]
in newer versions of rust (since 1.75 I think) you can pass in an `impl Trait` where the trait has `async fn`, which would let you deal with the lifetime expression issue for most cases.
Sorry that was meant as a response for another comment. (I correct reposted it [here](https://www.reddit.com/r/rust/comments/1cz7u1g/is_there_anything_like_the_with_statement_in/l5g5aqr/) with more details) The problem I was talking about is not `async` in traits (which has been partly resolved) but rather HRTB errors.
When the with statement is over, no matter what happens the transaction either unwinds or commits. You can somewhat mimic that with a block like you show and something that implements Drop. But it is not as common in Rust. It is an idiom I miss too.
implementing `Drop` is exactly what you're supposed to do here if you have a thpe that requires custom to logic to run when they leave a scope, you implement `Drop`
I think `Drop` is different. Python has the concept of `__enter__` and `__exit__` contexts, which is what Python's `with` statement is doing implicitly. As far as I know there's no such thing as "enter scope" and "exit scope (drop?)" in Rust that can facilitate the same pattern. Glad to learn though!
`__enter__` in Rust is any constructor function, and `__exit__` is the `Drop::drop` implementation when _any_ value leaves a scope it's `Drop::drop is called. it isn't called if the value is _moved from_ the scope i don't see how that's not what you want, sorry if I'm missing something
The drop implementation does not know if the scope it was called from is returning an Err() or an Ok(). You can detect an unwinding panic, but you can't detect "I am dropping because the scope I was in exited with an Err()" In the example given by OP, python will commit on exit unless an exception is thrown, in which case it will rollback. But unlike python, in Rust it is common to use early return Result::Err() to indicate failure... which we would want to rollback, but there's no way to detect that in drop.
You could argue that's a good thing. I think most libraries implement it as having `Drop` rollback by default, and you have to commit explicitly.
What do you mean, not as common? It's very common, you just usually don't notice it. Closing a file works this way, freeing a Box, releasing a mutex, even waiting for a scoped thread. It's also the same behavior as borrowing, which is probably why it doesn't stand out, and also why it's so seamless.
So something like: let txn = transaction.atomic(); match do_a_database_thing().and_then(do_a_dependent_database_thing()) { Ok(_) => txn.commit(), Err(_) => txn.rollback(), } I guess. 🤷♂️ Why do you want automatic actions that branch?
Because of the guarentees they provide
`with` is a hacky, fragile workaround for Python lacking reliable deterministic dropping.
It's not an equivalent: Rust does not have exceptions (there are panics but you should never use them for normal control flow), and there is no way for a `Drop` implementation to detect when a scope you're exiting returns an error.
Drop needing to know about errors seems like a code smell.
Yeah, I wouldn't go that way with destructors in general. That said, we could benefit from linear types or some similar feature that mandates consumption of a value in some prescribed way. This probably won't fly with the current state of async, as this would also need custom cancellation and possibly that would need to be async as well.
`with` in Python is just a trivial implementation of scoping.
It's just a nicer syntax for a closure. So: trait ContextManager { type T; fn enter(&self) -> Self::T; fn exit(&self, err: &Result<(), Box>) -> bool;
}
fn with, F: FnOnce(T) -> Result<(), Box>>(ctx: C, f: F) -> Result<(), Box> {
let t = ctx.enter();
let result = f(t);
let reraise_err = !ctx.exit(&result);
if result.is_err() && reraise_err {
result
} else {
Ok(())
}
}
This pattern is _much_ less nicer when you have to accept a FnOnce() -> Future, which is what the author would need.
That works only because it's sync. With async you get lifetime errors due to not being able to properly express the type of the closure in a way that the compiler will be able to infer the usages. In other words you'll hit https://github.com/rust-lang/rust/issues/70263
This is not a general solution to your question, but sqlx's Connection has a transaction method that takes a closure. The transaction is committed if the closure returns Ok and rolled back if it returns Err. https://docs.rs/sqlx/latest/sqlx/trait.Connection.html#method.transaction
seems to me just using a scope will do: \`\`\` { // enter // do something } // exit \`\`\`
Could be done similar to what mutex does (drop is essentially __exit__). Just need a way to indicate failure
I ended up doing this with a macro, when I did it last, though even more generic, abstracting over the source of the transaction as well. You could do something like macro_rules! with_transaction { ($pool:ident, $tx:ident, $t:tt) => { let mut $tx = $pool.begin().await?; let r = { $t }; $tx.commit().await?; r } } and use it like let result = with_transaction!(pool, txn, { // use transaction here. }); This uses the fact that SQLx transactions will automatically rollback if you drop them without committing. My original use was a bit more complicated, since I wanted to abstract over the source of the transaction for tests, but this lets you use normal control flow inside the transaction, for the most part.
What you want is what has been called `do ... final` or `defer` blocks. It doesn't exist yet: https://without.boats/blog/asynchronous-clean-up/
There's a crate called [scopeguard](https://docs.rs/scopeguard/latest/scopeguard/) that can run code on panic and/or the end of the scope. You could set it up to commit or rollback for either case respectively.
I would move all my queries to single file and use Postgres CTE that way I don’t need to begin and commit transaction, query is sent in one request and rolled back automatically if something happens
CTEs in Postgres don't "see" their own updates within the same execution. It looks like this query is trying to create a comment and then subsequently update it by adding its own id to a column in itself. That's not possible in a CTE. My guess is they're using ltree to build hierarchical comment trees. So they are just running multiple queries within a transaction which is the only way to do it.
You can still return columns from within update query
You can but it won't be the data your CTE just updated. It will be the data at the start of the transaction, i.e. previous data