Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
E
Ephemeral
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Volkor Barbarian Warrior
Ephemeral
Commits
7bd48be8
Unverified
Commit
7bd48be8
authored
2 years ago
by
Volkor Barbarian Warrior
Browse files
Options
Downloads
Patches
Plain Diff
speed up DB by like 2x at least by using WAL
parent
adaf5956
No related branches found
Branches containing commit
No related tags found
No related merge requests found
Changes
6
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
.gitignore
+3
-1
3 additions, 1 deletion
.gitignore
README.md
+2
-0
2 additions, 0 deletions
README.md
docs/benchmarks.md
+12
-0
12 additions, 0 deletions
docs/benchmarks.md
schema.sql
+5
-0
5 additions, 0 deletions
schema.sql
src/db.rs
+1
-1
1 addition, 1 deletion
src/db.rs
src/main.rs
+1
-1
1 addition, 1 deletion
src/main.rs
with
24 additions
and
3 deletions
.gitignore
+
3
−
1
View file @
7bd48be8
...
...
@@ -2,4 +2,6 @@
files.db
/files/
ephemeral.db
.timetracker
\ No newline at end of file
.timetracker
ephemeral.db-shm
ephemeral.db-wal
This diff is collapsed.
Click to expand it.
README.md
+
2
−
0
View file @
7bd48be8
...
...
@@ -26,8 +26,10 @@ That's a little more complicated.
Eventually we'll have 2 different environments, development and production, each with different logging and such.
1.
Initialise the database with the following command
`sqlite3 ephemeral.db < schema.sql`
1.
1. If you want to speed up sqlite a little bit more, run
`sqlite3 ephemeral.db`
and then
`PRAGMA synchronous = OFF;`
and then Ctrl+D
2.
Compile and run ephemeral with
`LOG=debug cargo run`
for the debug build.
`LOG=info cargo run -r`
will run the release build.
### Configuration Options
Configuration is done by settings environment variables in the launch command.
...
...
This diff is collapsed.
Click to expand it.
docs/benchmarks.md
+
12
−
0
View file @
7bd48be8
...
...
@@ -16,8 +16,20 @@ Benchmarking this software will be compared with QuadFile, the direct python equ
### Serving index
macro serving - 27k
### Uploading a file
macro upload - 42/s
### Serving a file
macro serving - 48/s
( with a few 5xx's)
### Deleting a file
## Notes
Changing from functions to macros didn't really speed anything up
is it an I/O bottleneck?
This diff is collapsed.
Click to expand it.
schema.sql
+
5
−
0
View file @
7bd48be8
-- Use Write-Ahead-Logging --
-- This speeds up queries by about 2x --
PRAGMA
journal_mode
=
WAL
;
-- PRAGMA synchronous = OFF; -- Disabled by default, can corrupt db if the computer suffers a catastrophic crash (or power failure)
-- This drops the existing files table, and re-creates it. --
DROP
TABLE
IF
EXISTS
'files'
;
CREATE
TABLE
IF
NOT
EXISTS
'files'
(
...
...
This diff is collapsed.
Click to expand it.
src/db.rs
+
1
−
1
View file @
7bd48be8
use
std
::
time
::
SystemTime
;
use
sqlx
::{
sqlite
::
SqliteQueryResult
,
Pool
,
Row
,
Sqlite
};
use
sqlx
::{
sqlite
::
SqliteQueryResult
,
Pool
,
Sqlite
};
// Adding a file to the database
// TODO: Fix panic on fileadd with same filename (even if isDeleted) (UNIQUE constraint)
...
...
This diff is collapsed.
Click to expand it.
src/main.rs
+
1
−
1
View file @
7bd48be8
...
...
@@ -13,7 +13,7 @@ use rand::Rng;
use
std
::
fs
::
create_dir_all
;
use
std
::
path
::
Path
;
use
std
::
time
::{
Duration
,
SystemTime
};
use
std
::
{
env
,
fs
}
;
use
std
::
fs
;
use
tokio
::{
task
,
time
};
use
tracing_subscriber
::
filter
::
EnvFilter
;
use
tracing_subscriber
::
fmt
;
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment