[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): table suspension reason in wal_tables() and warnings endpoint #4607

Merged
merged 45 commits into from
Jul 15, 2024

Conversation

glasstiger
Copy link
Contributor
@glasstiger glasstiger commented May 28, 2024

required by https://github.com/questdb/questdb-enterprise/pull/439 and questdb/ui#291


Adds two new columns to the output of wal_tables(): errorTag, errorMessage
These columns are populated only if the table is suspended.
They help to identify the reason of the table suspension.

errorMessage - the error message logged by QuestDB
errorTag - identifies the type of error, such as DISK FULL or TOO MANY OPEN FILES

The error tag will help the Web Console to direct the user to a page with instructions how to resolve the issue.


Also adds a new endpoint: /warnings.
This returns with a json object containing server configuration issues.
The web console calls this on every reload, and displays the warnings.

Currently we are checking for 3 configuration issues: UNSUPPORTED FILE SYSTEM, TOO MANY OPEN FILES and OUT OF MMAP AREAS.


Also fixes error codes in CairoException.
The same error codes were used for different errors.


Also adds a new SQL command to be able to suspend tables.
Optionally an error code/tag and error message can be specified.
The command has been added primarily to help testing.
Examples:

ALTER TABLE tab1 SUSPEND WAL;
ALTER TABLE tab1 SUSPEND WAL WITH 24, 'test error';
ALTER TABLE tab1 SUSPEND WAL WITH 'DISK FULL', 'test error';

Also adds a new SQL function to simulate warnings.
Examples:

select simulate_warnings('UNSUPPORTED FILE SYSTEM', 'Unsupported file system [dir=/questdb/path/dbRoot, magic=0x6400A468]');
select simulate_warnings('TOO MANY OPEN FILES', 'fs.file-max limit is too low [current=1024, recommended=1048576]');
select simulate_warnings('OUT OF MMAP AREAS', 'vm.max_map_count limit is too low [current=4096, recommended=1048576]');

Web console should be reloaded after the function is executed, and the warning should appear.
Warnings accumulate, so you should see as many warnings as commands issued.
You can clear the warnings by running:

select simulate_warnings('', '');

@glasstiger glasstiger marked this pull request as draft May 28, 2024 09:28
@glasstiger glasstiger marked this pull request as ready for review May 28, 2024 23:19
@bluestreak01
Copy link
Member

can we add a SQL function, which would suspend any table with the given message and error code? Otherwise UX is very hard to test. We already have a debug function, such as SimulateCrashFunctionFactory. You can use it for an inspiration

@glasstiger
Copy link
Contributor Author

can we add a SQL function, which would suspend any table with the given message and error code? Otherwise UX is very hard to test. We already have a debug function, such as SimulateCrashFunctionFactory. You can use it for an inspiration

added a new ALTER command to suspend tables.

@glasstiger glasstiger changed the title feat(core): display table suspension reason in wal_tables() feat(core): display table suspension reason in wal_tables() and add new endpoint for warnings Jul 11, 2024
@glasstiger glasstiger changed the title feat(core): display table suspension reason in wal_tables() and add new endpoint for warnings feat(core): table suspension reason in wal_tables() and warnings endpoint Jul 11, 2024
nwoolmer
nwoolmer previously approved these changes Jul 15, 2024
Copy link
Contributor
@nwoolmer nwoolmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, checked the new syntax/functions, and that the config option works.

@glasstiger glasstiger added SQL Issues or changes relating to SQL execution REST API Issues or changes relating to the HTTP endpoints labels Jul 15, 2024
@ideoma
Copy link
Collaborator
ideoma commented Jul 15, 2024

[PR Coverage check]

😍 pass : 252 / 284 (88.73%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/DefaultCairoConfiguration.java 0 1 00.00%
🔵 io/questdb/griffin/engine/functions/rnd/RndLogFunctionFactory.java 2 9 22.22%
🔵 io/questdb/griffin/engine/functions/catalogue/SimulateCrashFunctionFactory.java 10 12 83.33%
🔵 io/questdb/griffin/SqlCompilerImpl.java 40 48 83.33%
🔵 io/questdb/griffin/engine/functions/catalogue/SimulateWarningsFunctionFactory.java 16 19 84.21%
🔵 io/questdb/cairo/ErrorTag.java 27 32 84.38%
🔵 io/questdb/cairo/wal/ApplyWal2TableJob.java 33 36 91.67%
🔵 io/questdb/cutlass/http/processors/WarningsProcessor.java 62 65 95.38%
🔵 io/questdb/griffin/engine/functions/test/TestNPEFactory.java 7 7 100.00%
🔵 io/questdb/PropServerConfiguration.java 3 3 100.00%
🔵 io/questdb/cairo/wal/seq/SeqTxnTracker.java 11 11 100.00%
🔵 io/questdb/cutlass/http/processors/JsonQueryProcessor.java 1 1 100.00%
🔵 io/questdb/cutlass/http/HttpServer.java 4 4 100.00%
🔵 io/questdb/std/FilesFacadeImpl.java 3 3 100.00%
🔵 io/questdb/griffin/CompiledQueryImpl.java 2 2 100.00%
🔵 io/questdb/PropertyKey.java 1 1 100.00%
🔵 io/questdb/griffin/engine/functions/catalogue/WalTableListFunctionFactory.java 20 20 100.00%
🔵 io/questdb/cairo/wal/seq/TableSequencerAPI.java 1 1 100.00%
🔵 io/questdb/cairo/CairoConfigurationWrapper.java 1 1 100.00%
🔵 io/questdb/griffin/SqlKeywords.java 8 8 100.00%

Copy link
Contributor
@nwoolmer nwoolmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@glasstiger
Copy link
Contributor Author

Looks good!

Thank you for the review!

@bluestreak01 bluestreak01 merged commit 22d99f0 into master Jul 15, 2024
28 checks passed
@bluestreak01 bluestreak01 deleted the ia_wal_error branch July 15, 2024 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
REST API Issues or changes relating to the HTTP endpoints SQL Issues or changes relating to SQL execution tandem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants