apiserver does not close rows in InitializeDBStatusTable leading to "busy buffer" errors

[discordianfish]

What steps did you take:

Not exactly sure how the apiserver got into this state, probably something went wrong on initial db creation.

What happened:

ml-pipeline deployment is crashlooping:

I0217 11:55:37.974326       1 client_manager.go:132] Initializing client manager
I0217 11:55:37.974482       1 config.go:45] Config DBConfig.ExtraParams not specified, skipping
[mysql] 2020/02/17 11:55:38 packets.go:427: busy buffer
[mysql] 2020/02/17 11:55:38 packets.go:408: busy buffer
E0217 11:55:38.244000       1 default_experiment_store.go:73] Failed to commit transaction to initialize default experiment table
[mysql] 2020/02/17 11:55:38 packets.go:427: busy buffer
[mysql] 2020/02/17 11:55:38 packets.go:408: busy buffer
E0217 11:55:38.253882       1 db_status_store.go:71] Failed to commit transaction to initialize database status table
[mysql] 2020/02/17 11:55:38 packets.go:427: busy buffer
[mysql] 2020/02/17 11:55:38 packets.go:408: busy buffer

What did you expect to happen:

Database creation should work.

Environment:

KFP version: 1.0-rc

Anything else you would like to add:

I suspect the root issue here is that the apiserver re-uses the transaction before closing the rows, which according to go-sql-driver/mysql#977 (comment) would cause such 'busy buffer' error.

Similar/same? issues: #1745 and #2699

/kind bug
/area backend

I'll submit a PR to fix in a bit.

[jingzhang36]

@discordianfish Thank you very much for the fix. Just one thing: do you mind adding a comment to the rows.close() line, stating that the variable "rows" shouldn't be used after that point (i.e., after it is closed). I am thinking of later updates to these two files accidentally referring to "rows" after it is closed and causes errors. Thanks!

[discordianfish]

@jingzhang36 Okay, I thought that goes without saying but I've added a comment.