Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #543 +/- ##
===========================================
+ Coverage 99.17% 99.19% +0.02%
===========================================
Files 146 147 +1
Lines 7400 7493 +93
===========================================
+ Hits 7339 7433 +94
+ Misses 61 60 -1
... and 5 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
Hi! Thanks for the PR. I think we need to do some redesigning to make this cleaner. First, the Second, I see some 'smells' with the current implementation:
There is also an I don't see a perfect answer, but if I were writing this, I'd try this design and see if it looks clean:
If you can think of an alternative proposal, I'm happy to discuss. Best, |
|
Your design suggestion looks clean for me. I will merge We still need to expose If Thanks. |
|
Storing a reference should be fine and looks good practice. Please add UTs for these, it looks like I skipped them. Thank you. |
|
Ready for review. Additionally, rearranged some fields - |
|
Thanks. Will have a look tomorrow. |
|
While I was doing the review (comments coming soon), I encountered this (new) potential pitfall: Imagine you're using the multi-function API. You get rows with
Before this PR, the following used to work because serialization happened before the read buffer was touched at all: execution_state st;
// start_execution, read more rows...
auto rows = conn.read_some_rows(st);
conn.execute(with_params{"SELECT ... {}"}, rows.at(0).at(0).as_string());Now this is UB, because So before going on with this, we need to make a decision. I see two options:
Honestly, I don't think this use case is very common. Before I go on with the review, what do you think of this? |
|
This is a good catch, indeed. I have some thoughts on a possible compromise solution: could we just change return type of // Dangling Pointer
std::string_view sv = rows.at(0).at(0).as_string();But it's just a random idea that occurred to me and I haven't really given it much thought. Regarding your suggestions, I don't think merging buffers is worth the breaking change. If you want to revert this change, I would not be against it. But if we choose to support such usage of the multi-function API, we should cover this specific case to avoid breaking it in the future. Thanks. |
That's a dangerous change, yes. I wouldn't go down this route. At the moment, the recommended way to get an owning type would be using
I complete agree with this. I've been trying to compose an example reproducing the problem, but I'm failing to see a plausible case where this breaks and that is not highly dubious to begin with. For instance: mysql::execution_state st;
co_await conn.async_start_execution("SELECT first_name, last_name, salary FROM employee WHERE id = 42", st);
mysql::rows_view employees = co_await conn.async_read_some_rows(st);
assert(!st.shoud_read_rows());
co_await conn.async_execute(with_params("... {}", employees.at(0).at(0)), some_result);This works most of the time when you have only 1 row, but is not guaranteed to, because the EOF packet is not guaranteed to be read in the same batch as the row (it likely is, but it may not, depending on the network and the row size). Another try: mysql::rows_view employees;
do
{
mysql::rows_view employees = co_await conn.async_read_some_rows(st);
for (auto employee : employees)
print_employee(employee);
} while (st.should_read_rows());
if (!employees.empty())
co_await conn.async_execute(mysql::with_params("... {}", employees.at(0).at(0)), some_result);This doesn't trigger the assert, but once again, you don't have any guarantee that this So: can you think of a use case that is plausible and actually breaks? I'm a bit divided. On the one hand, keeping it as-is is the safest, and the gains aren't that big. On the other hand, I don't want to encourage use cases that aren't sound at all. |
|
I have researched and haven't found any guaranteed way to fetch a row with an EOF packet over TCP even if we control row size with query When using std::vector<std::string> names;
do
{
mysql::rows_view employees = co_await conn.async_read_some_rows(st);
for (auto employee : employees)
names.push_back(employee.at(0));
} while (st.should_read_rows());
// Process received names with async_execute() |
Yes. The docs state it.
Back when I wrote it, I envisioned it for large resultsets within ETL processes. Imagine you want to write your rows into a file: do
{
mysql::rows_view employees = co_await conn.async_read_some_rows(st);
for (auto employee : employees)
// write the employee to the file, or to a socket, database, HTTP connection...
} while (st.should_read_rows()); |
In that case where large rows are consumed inside of a loop and copying is expensive, I would rather use connection pool if I need to write to database. Using last received And the problem stays the same: there is no guaranteed way to reproduce breaking use case, especially for a big row. |
|
Decided: we proceed with this. I'd ask you to slightly change |
close #279
Because
std::vectorsize is managed manually inread_bufferand never shrinks, we cannot directly change its size (with clear()) becauseread_bufferrelies onstd::vectorstate, thus we need to use another wrapper, which manages size manually just asread_buffer. Furthermore, we usenext_power_of_twoto grow.There is no
max_buffer_size_forbuffer_writerbecause such behaviour already implemented inserialization_context.