aboutsummaryrefslogtreecommitdiff
path: root/TODOs.org
blob: acdc2f37708209d25b86b0a37cedcd5e20150233 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
* Tasks
:PROPERTIES:
:CUSTOM_ID: tasks
:END:
** TODO Duplication of calls to =reduce=
:PROPERTIES:
:CUSTOM_ID: ce335932-dc91-4871-be14-5724e0c451ec
:END:
- State "TODO"       from              [2020-10-11 dim. 08:05]

There's probably something to be abstracted away there.
** TODO EDN parsing crate [0/14]
:PROPERTIES:
:CUSTOM_ID: bcaab441-c562-4baf-b2ac-fcf8e2e761f4
:END:
- State "TODO"       from              [2020-10-02 ven. 13:35]

Implement the "edn" crate (under =src/lib/edn/rust/=), with the 3 expected
artifacts: Rust library, C FFI and WASM.

Missing things to be done before calling the crate complete:
- [ ] symbol grammar
- [ ] number grammar
- [ ] add =description= fields to =EDNResult= struct
- [ ] test with quick check
- [ ] intern symbols and keywords
- [ ] support pretty-printing
- [ ] handle =\n= same as a return
- [ ] create builtint tagged elements: =#uuid ""= and =#inst ""=
- [ ] allow extensible tagged elements (use a =IEDNValue= trait)
- [ ] hashable version of =HashMap= and =HashSet=

After the Rust code is done, we still need to generate the bindings for the
languages that we want to use:
- [ ] TypeScript
- [ ] +Swift+
- [ ] Kotlin
- [ ] Dart
The prototype generator is under =src/lib/x-bindgen/=.
** TODO Unify gitignore snippets in default.nix with in .gitignore
:PROPERTIES:
:CUSTOM_ID: 335a5f23-c79c-4df6-bc4b-7f19dafbfbe8
:END:
- State "TODO"       from              [2020-10-02 ven. 15:06]
** DONE Add better folder structure to accomodate dependencies inside the repo :sdks:
CLOSED: [2020-10-02 ven. 21:24]
:PROPERTIES:
:CUSTOM_ID: 68e6f839-2df6-4f73-82e3-5aa73d33b580
:END:
- State "DONE"       from "TODO"       [2020-10-02 ven. 21:24] \\
Everything now lives under =src/=, with 3 subdirectories, =core/=, =sdks/= and
=libs/=.
- State "TODO"       from              [2020-10-02 ven. 11:46]

Right now I want to add two more "categories" of source code to the repository:
"aux" and "lib". This is a bad idea, I should figure out a better way to
organize this.
** TODO Build SDKs using an static lib instead of a dynamic one        :sdk:
:PROPERTIES:
:CUSTOM_ID: b64a2a62-4ded-449d-a672-ac54b158f3b3
:END:
- State "TODO"       from              [2020-10-02 ven. 08:21]

This way downstream users can include this single binary dependency that is self contained.
** TODO Is the =#[derive(Debug)]= stripped out in release compilations?
:PROPERTIES:
:CUSTOM_ID: 7cce6ced-315a-41c6-99d3-a462a7e95f03
:END:
- State "TODO"       from              [2020-10-01 jeu. 08:31]
** DONE =assert-metadata.sh=: Forbid dependencies with different versions on Rust crates :dependencies:
CLOSED: [2020-10-02 ven. 11:32]
:PROPERTIES:
:CUSTOM_ID: 0503f875-f32c-4988-a03d-d36174d05aed
:END:
- State "DONE"       from "TODO"       [2020-10-02 ven. 11:32] \\
Done in {{{commit(c7fdd86a95bd6d4fea934f8a4bc6bcfa41600a29)}}}.
- State "TODO"       from              [2020-09-30 mer. 19:41]
** TODO Stop using FOD for Kotlin
:PROPERTIES:
:CUSTOM_ID: b5ec0772-d567-4c82-9671-251e2997b5f2
:END:
- State "TODO"       from              [2020-09-30 mer. 18:01]
Use a tool like =mavenix= or similar to better resolve dependencies.
** TODO Run Rust tests with Cargo
:PROPERTIES:
:CUSTOM_ID: 1542b395-d685-46d9-88a3-bdb4cb72e794
:END:
- State "TODO"       from              [2020-09-30 mer. 18:02]

Instead of naersk running them at build time, they should be ran inside the
=./tests.sh= file itself.
** DONE Add =sphinx-intl= to nixpkgs upstream
CLOSED: [2020-10-04 dim. 09:00]
:PROPERTIES:
:CUSTOM_ID: b6f62adf-db5e-4790-b101-b86ca9265a51
:END:
- State "DONE"       from "TODO"       [2020-10-04 dim. 09:00] \\
Done in pull request [[https://github.com/NixOS/nixpkgs/pull/99544][#99544 on GitHub]].
- State "TODO"       from              [2020-09-27 dim. 14:15]

Code is available on a [[https://github.com/NixOS/nixpkgs/compare/master...EuAndreh:sphinx-intl?expand=1][nixpkgs fork]]. All is left to do is perform the steps
described in the CONTRIBUTING of the nixpkgs repo and create the pull request.
** TODO FFI [3/10]
:PROPERTIES:
:CUSTOM_ID: e0003400-2e37-4f1e-9417-d43b830983b7
:END:
- State "TODO"       from              [2020-09-26 sam. 19:15]

Task to track the evolution of FFI interop over SDKs.

Calling Rust from $LANGUAGE:
- [ ] browser (TypeScript)
- [X] Node.js (TypeScript)
- [X] Swift
- [ ] Kotlin
- [X] Dart

Calling $LANGUAGE from Rust:
- [ ] browser (TypeScript)
- [ ] Node.js (TypeScript)
- [ ] Swift
- [ ] Kotlin
- [ ] Dart
** TODO Bottom-up POC [0/5]                                            :poc:
:PROPERTIES:
:CUSTOM_ID: 30a2d6a5-e4d7-477f-ad4d-355ed27e52af
:END:
- State "TODO"       from              [2020-09-20 dim. 18:27]

- [ ] FFI + SDKs (with tests)
- [ ] SQLite over FFI on SDKs (with tests)
- [ ] WASM + SDK (with tests)
- [ ] IndexedDB over WASM on SDK (with tests)
- [ ] Nix
** TODO Top-down POC [0/6]
:PROPERTIES:
:CUSTOM_ID: 9b8b8fbd-c923-4343-8389-1906947dca0d
:END:
- State "TODO"       from              [2020-09-27 dim. 18:29]

Clojure reference top-down implementation.
- [ ] EAVT index
- [ ] IMergeStrategy
- [ ] AVET
- [ ] Datalog
- [ ] Pull
- [ ] Full-text search
** TODO Merge strategy [0/3]                                           :poc:
:PROPERTIES:
:CUSTOM_ID: f4755a97-59aa-496c-8b25-754ad80a8807
:END:
- State "TODO"       from              [2020-09-20 dim. 21:31]

- [ ] =:db.conflict-resolution/none=
- [ ] =:db.conflict-resolution/last-write-wins=
- [ ] =:db.conflict-resolution/automerge= to support [[https://doi.org/10.1016/j.jpdc.2010.12.006][commutative text editing]]
  using a [[https://github.com/automerge/automerge-perf/tree/master/columnar][binary representation]] of the CRDT. This is less about supporting this
  specific algorithm and more about putting the versatility of the building
  blocks to a test, and see if they are working well at the right level
** TODO Remove duplicates (from capitalization) from dictionaries     :docs:
:PROPERTIES:
:CUSTOM_ID: 2b4158bc-ad02-4fb5-ab35-bd3e86602128
:END:
- State "TODO"       from              [2020-09-20 dim. 01:02]

See http://www.openoffice.org/lingucomponent/affix.readme,
https://www.suares.com/?page_id=25&news_id=233
** TODO Add anchors to glossary terms                                 :docs:
:PROPERTIES:
:CUSTOM_ID: 901af2bb-9b7c-4fa1-a50d-68cd9f7fdbc8
:END:
- State "TODO"       from              [2020-09-07 lun. 15:37]

This way, one can share the link to a specific term without asking the other
person to "search the glossary page for X".

One can accomplish that by going to the index, but that's hidden.
** TODO A better wording for =embedded=                               :docs:
:PROPERTIES:
:CUSTOM_ID: f995a42b-049e-405c-a9ef-5a551b827140
:END:
- State "TODO"       from              [2020-09-20 dim. 17:09]
People usually think about embedded systems (as in Arduino) instead of
client-side, local-first things.

Maybe the glossary entry is enough, but the confusion points to a need of a
better word, but I couldn't find any yet.
** TODO Generate language bindings instead of writing them manually [1/6] :ffi:
:PROPERTIES:
:CUSTOM_ID: 3c64ed93-a884-46f5-b2e4-c0afe44b2e85
:END:
- State "TODO"       from              [2020-09-25 ven. 17:01]

- [ ] browser
- [ ] Node.js
- [X] +Swift+ it already knows how to read C the header directly, nothing needs
  to be generated
- [ ] Kotlin
- [ ] Dart

I didn't expect me to say this, but good job Swift. It doesn't even need
something like =rust-bindgen= to parse header files and generate +Rust+ Swift
code, it just includes the header file. Even though Rust adds no runtime
overhead to doing FFI, it does handle C at an arms length, and this shows up
when a =rust-bindgen= tool is required.

I'm not sure if there is any internal translation for Swift, though. It doesn't
seem to have, given the integration with C is so tight.
** TODO Build and test isolation with Nix [6/7]                         :ci:
:PROPERTIES:
:CUSTOM_ID: 0b6d9623-56f7-47d7-b4c8-8d1ce094126d
:END:
- State "TODO"       from              [2020-09-27 dim. 15:52]

Instead of using Nix just to build the environment, use it to build to enforce
build isolation, like how the documentation works. Leverage tools like
=crate2nix= and =node2nix=.

I worked on this a bit before, but the code for it was removed in
{{{commit(8896733493660a7bff5b1b58d9e111a4b5e9656f)}}}.

Setup $LANGUAGE with Nix:
- [X] Clojure - done in {{{commit(6e074faf626c355d9d5904fc25126ecd6c8f4f5c)}}}
- [X] Rust - done in {{{commit(d2f271f83096092c43c129bde10d8527dee35cbe)}}}
- [X] browser (TypeScript) - done in
  {{{commit(11ea3e6d5b492feeee2da7c5dd67bd75abfae562)}}}
- [X] Node.js (TypeScript) - done in
  {{{commit(4de284ef66a94321af66f21f2674bbeaacdef960)}}}
- [X] Swift - all it required were the headers from .envrc to be available
  inside nix-build
- [X] Kotlin - done in {{{commit(6f718036aaca04573bd3da5d7344f22d5329085b)}}}
- [ ] Dart
** TODO Increase Kotlin JDK version from 8                    :dependencies:
:PROPERTIES:
:CUSTOM_ID: f4bf9bd2-4600-4d9e-aaea-8e36b0420f10
:END:
- State "TODO"       from              [2020-09-27 dim. 10:33]
** TODO Increate Node.js version from 10                      :dependencies:
:PROPERTIES:
:CUSTOM_ID: c8d6e427-ae25-4b89-b39c-71b3e80ee170
:END:
- State "TODO"       from              [2020-09-27 dim. 16:12]
** TODO Look at =spec-alpha2=                                          :poc:
:PROPERTIES:
:CUSTOM_ID: 2bffeee1-3fbf-4642-ab0b-afca3bca0f99
:END:
- State "TODO"       from "CANCELLED"  [2020-09-27 dim. 19:09] \\
Now the Clojure POC became the reference implementation, so it will live much
longer than I thought when closing this task.
- State "CANCELLED"  from "TODO"       [2020-09-20 dim. 14:21] \\
I'm not working on the Clojure POC anymore.
- State "TODO"       from              [2020-09-08 mar. 13:51]

It's said to have better tools for dynamic spec definition.
** TODO Keep =build.gradle.kts= and =pom.xml= in sync
:PROPERTIES:
:CUSTOM_ID: 793cadea-09b0-435c-b63f-15bdea2b09e7
:END:
- State "TODO"       from              [2020-09-28 lun. 11:00]
Right now they have to be kept consistent manually.

Maybe read from a JSON file in =build.gradle.kts= and generate the =pom.xml=
file from the same JSON file.
** DONE Is having both a =ffi= and an =api= crates redundant?    :bottom_up:
CLOSED: [2020-09-30 mer. 18:46]
:PROPERTIES:
:CUSTOM_ID: cf4f154a-5ebc-4e33-854f-429a57019731
:END:
- State "DONE"       from "TODO"       [2020-09-30 mer. 18:46] \\
Yes ☺️. I can have a single crate with both =cdylib= and =lib= as =crate-type=s.
The =lib= =crate-type= allows it to be importable from other Rust code directly,
without going through FFI.
- State "TODO"       from              [2020-09-30 mer. 18:18]

Is it better to duplicate the functions from =api= crate in =ffi= crate, or
should I just stick to the =ffi= crate?
** DONE Can the WASM API be built from the same =api= crate?
CLOSED: [2020-10-02 ven. 11:41]
:PROPERTIES:
:CUSTOM_ID: cf3a7f4c-619f-428b-9851-7c0730f07f22
:END:
- State "DONE"       from "TODO"       [2020-10-02 ven. 11:41] \\
Yes, both for "mediator" and "edn" crates. "eq" and "cli" are desktop only (FFI
and Rust artifacts, no WASM build involved).

Done in {{{commit(18fb2e052590197114ed5cc0cf286a68db9138d6)}}}.
- State "TODO"       from              [2020-09-30 mer. 19:22]

As in {{{task(cf4f154a-5ebc-4e33-854f-429a57019731)}}}, can the =wasm= crate be
merged with the =api= crate?
** DONE Is having both =#[no_mangle]= and =extern "C"= redundant? :bottom_up:
CLOSED: [2020-09-30 mer. 19:30]
:PROPERTIES:
:CUSTOM_ID: ebe685a1-8ed0-4d99-8b68-572cd577e9b1
:END:
- State "DONE"       from "TODO"       [2020-09-30 mer. 19:30] \\
It isn't. Without =#[no_mangle]= the =pub extern "C" fn foo= isn't added to the
generated cbindgen header file. This might be a detail about cbindgen, and I
can't tell the difference yet.
- State "TODO"       from              [2020-09-30 mer. 18:23]

Does the =extern "C"= declaration already prevents the names from being mangled?
** DONE Remove duplication of metadata                                :repo:
CLOSED: [2020-09-27 dim. 19:10]
:PROPERTIES:
:CUSTOM_ID: 5ebe4840-53bd-421c-95e0-4f67d68fd7b5
:END:
- State "DONE"       from "TODO"       [2020-09-27 dim. 19:10]
- State "TODO"       from              [2020-09-22 mar. 19:33]

A.K.A. "revamp assert-metadata.sh".

Project metadata is spread everywhere: every =Cargo.toml= file, =package.json=
etc. Those should all come from a single place, probably =metadata.json=.
** DONE Add english hunspell validation to the built documentation    :docs:
CLOSED: [2020-09-20 dim. 01:03]
:PROPERTIES:
:CUSTOM_ID: 4005a8cf-dbf6-4640-9529-3d1d5a273529
:END:
- State "DONE"       from "TODO"       [2020-09-20 dim. 01:03]
- State "TODO"       from              [2020-09-07 lun. 07:01]
** DONE Extend existing glossary terms beyond mere stubs              :docs:
CLOSED: [2020-09-20 dim. 17:12]
:PROPERTIES:
:CUSTOM_ID: a0e1266a-17cc-497b-8801-e13862903bfa
:END:
- State "DONE"       from "TODO"       [2020-09-20 dim. 17:12] \\
Done in {{{commit(6dcdace27caa833445bb419a45a786386b387d3d)}}}.
- State "TODO"       from              [2020-09-07 lun. 16:50]
** CANCELLED Add search capabilities to the tasks-and-bugs page
CLOSED: [2020-09-19 sam. 19:29]
:PROPERTIES:
:CUSTOM_ID: 1ac5fda6-da43-4103-956f-6dee25a5c56b
:END:
- State "CANCELLED"  from "TODO"       [2020-09-19 sam. 19:29] \\
The best search is CTRL+f.
- State "TODO"       from              [2020-09-08 mar. 06:58]

/Pilfer/ syntax [[https://issues.guix.gnu.org/help#search][Guix issue tracker]], and implement it as a minimalistic
JavaScript visibility toggle of DOM elements.
* Bugs
:PROPERTIES:
:CUSTOM_ID: bugs
:END:
** DONE Support tags in tasks-and-bugs
CLOSED: [2020-09-20 dim. 12:06]
:PROPERTIES:
:CUSTOM_ID: 327ad6a1-e867-4e17-9196-2a243d51c4cd
:END:
- State "DONE"       from "TODO"       [2020-09-20 dim. 12:06] \\
Done in {{{commit(68e2a5c63e3fa8a83fc22652c1e36e9df4492d26)}}}.
- State "TODO"       from              [2020-09-08 mar. 07:12]
** TODO Fix date in documentation build
:PROPERTIES:
:CUSTOM_ID: 81f951f6-e669-42dc-8803-fc7613cda978
:END:
- State "TODO"       from              [2020-09-20 dim. 17:17]

PDF date is set to January 1st, 1980, and ePUB errors with:
: ValueError: ZIP does not support timestamps before 1980
* Improvements
** Add C++ SDK?
Pros:
C++ is a language that's also used on cross-platform development.
Cons:
Yet another SDK to support.
https://git.sr.ht/~sircmpwn/wio
** Add C, C++, Objective-C code examples
* Questions
** Why am I adding the =timeline-id= to the transaction?
* Resources
** CRDT
*** [[https://josephg.com/blog/crdts-are-the-future/][I was wrong. CRDTs are the future]]
** C
*** [[http://libcello.org/][Cello High Level C]]
*** AFL fuzzer
http://lcamtuf.coredump.cx/afl/
https://github.com/google/AFL
*** [[https://git.sr.ht/~sircmpwn/cstyle#features-to-avoid][C Style Guide]]
*** [[https://www.amazon.com/Advanced-Programming-Example-John-Perry/dp/0534951406][Advanced C Programming by Example]]
*** [[https://modernc.gforge.inria.fr/][Modern C]]
*** [[https://www.amazon.com/Programming-Language-2nd-Brian-Kernighan/dp/0131103628][C Programming Language]]
*** [[https://publications.gbdirect.co.uk//c_book/][The C Book]]
*** [[https://www.cprogramming.com/debugging/valgrind.html][Using Valgrind to Find Memory Leaks and Invalid Memory Use]]
** WASM
*** [[https://github.com/WebAssembly/design/issues/1233][WASI IndexedDB API]]
*** [[https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB][Using IndexedDB]] on MDN
*** [[http://web.archive.org/web/20190624214907/https://www.w3.org/TR/IndexedDB-2/][Indexed Database API 2.0]]
*** [[https://surma.dev/things/c-to-webassembly/][Compiling C to WebAssembly without Emscripten]]
*** [[https://surma.dev/things/raw-wasm/][Raw WebAssembly]]
*** [[https://github.com/WebAssembly/gc/blob/master/proposals/gc/Overview.md][WASM GC]]!!! wat?!!???
** Databases
*** [[https://www.amazon.com/Database-Design-Implementation-Data-Centric-Applications/dp/3030338355][Database Design and Implementation]]
Highly recommend by https://news.ycombinator.com/item?id=19583927

Also:
- https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/
- https://www.youtube.com/watch?v=F3XGUPll6Qs
- https://blog.marcua.net/post/117671929
- https://dsf.berkeley.edu/papers/fntdb07-architecture.pdf
- http://www.redbook.io/
*** [[https://db.in.tum.de/teaching/ss17/moderndbs/][Practical Course: Database Implementation]]
https://db.in.tum.de/teaching/ss17/moderndbs/
*** CMU Database Systems
*** Databases book
*** [[https://www.sqlite.org/books.html][SQLite Forensics]]
*** [[https://www.sqlite.org/books.html][SQLite Database System Design and Implementation]]
*** PostgreSQL [[https://www.postgresql.org/docs/books/][book]]s
*** [[https://jvns.ca/blog/2014/10/02/how-does-sqlite-work-part-2-btrees/][How does SQLite work? Part 2: btrees! (or: disk seeks are slow don't do them!)]]
*** [[https://www.amazon.com/Modern-B-Tree-Techniques-Foundations-Databases/dp/1601984820][Modern B-Tree Techniques]]
suggested by Andy Pavlo in lecture 7 "Tree Indexes I".
*** Dragon book
In the [[https://www.youtube.com/watch?v=Z_cX3bzkExE][SQLite talk by Richard Hipp]] he talks about how the process of parsing and executing the SQL query is exactly like compiling a program, and mentioned the dragon book.

In [[https://www.youtube.com/watch?v=P76Vbsk_3J0][Clojure for Java Programmers Part 1 - Rich Hickey]] (around ~30min), Rich Hickey talks about the duality of runtime and compilation time when talking about compiled regexes in Java.

Also, Clojure being compiled to JVM bytecode, which itself is intepreted by the JVM runtime, which is itself compiled by the JIT, reinforces this notion of compile-time/runtime duality.
*** SQLite source code
Also: ANSI C book
*** [[https://matt.sh/howto-c][How to C in 2016]]
*** [[https://sqlite.org/whyc.html][Why Is SQLite Coded In C]]
On SQLite usage of standard C library functions, like =memcpy= and =malloc=.
*** [[https://sqlite.org/opcode.html][The SQLite Bytecode Engine]]
*** [[https://sqlite.org/testing.html][How SQLite Is Tested]]
*** [[https://pingcap.com/blog/how-we-compiled-a-golang-database-in-the-browser-using-webassembly/][How We Compiled a Golang Database in the Browser Using WebAssembly]]
** Related databases to look more deeply into
*** [[https://opencrux.com/main/index.html][Crux]]
*** [[https://irmin.org/][Irmin]]
*** [[https://github.com/liquidata-inc/dolt][Dolt]]
*** [[https://fauna.com/][FaunaDB]]
*** [[https://github.com/google/lovefield/blob/master/docs/dd/04_query_engine.md][lovefield]]'s query engine
*** [[https://github.com/replikativ/datahike][datahike]]
From [[notmuch:id:87y2luzb9v.fsf@euandre.org][Email to James Conroy-Finn: Re: A database you might find interesting]].
*** [[https://github.com/TimelyDataflow/differential-dataflow][differential-dataflow]]
*** [[https://sled.rs/simulation][sled simulation engine]]
From: https://lobste.rs/s/m9vkg4/database_i_wish_i_had#c_askorb
*** [[https://github.com/tonsky/datascript][DataScript]]
*** CRR (email + PDF attachment)
From [[notmuch:id:87h7sihwc5.fsf@uit.no][Email from Weihai Yu: CRR (Re: The database I wish I had)]].
*** [[https://duckdb.org/][DuckDB]]
From [[notmuch:id:0A0E9455-BE36-4E2B-9041-CF954F8D8D09@cwi.nl][Email from Hannes Mühleisen: Embeddable databases]].
*** CRDTs
*** [[https://hackage.haskell.org/package/acid-state][acid-state]]
*** SQLCipher (email from @rnewman)
*** [[https://github.com/mikeal/dagdb][dagdb]]
*** [[https://jneem.github.io/merging/][Part 1: Merging and patches]]
*** [[https://github.com/jrhy/mast][mast]] and [[http://htmlpreview.github.io/?https://github.com/indeliblesystems/indelible/blob/master/doc/000-API-ref.html][Indelible]]
*** AllegroGraph
From [[notmuch:id:010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com][Email from Richard Newman: Re: The database I wish I had]].
*** [[https://dropbox.tech/developers/how-the-datastore-api-handles-conflicts-part-1-basics-of-offline-conflict-handling][Dropbox Datastore API]]
From [[notmuch:id:010101744f99448f-fa8dd88e-0daf-41f8-b3ca-6ea551c300e4-000000@us-west-2.amazonses.com][Email from Richard Newman: Re: The database I wish I had]].
*** [[https://github.com/attic-labs/noms][noms]]
*** [[https://dropbox.tech/developers/how-the-datastore-api-handles-conflicts-part-1-basics-of-offline-conflict-handling][How the Datastore API Handles Conflicts – Part 1: Basics of Offline Conflict Handling]]
*** [[https://cs-syd.eu/posts/2019-10-14-mergeful-value][Announcing mergeful, part 1: Cooperative agreement on a single value]]
*** [[https://jonas-schuermann.name/projects/dvcs-for-structured-data/blog/2020-03-22-manifesto.html][I Want Decentralized Version Control for Structured Data!]]
https://jonas-schuermann.name/projects/dvcs-for-structured-data/notes/references.html
*** https://gitlab.com/MazeChaZer/dvcs-for-structured-data/-/blob/master/source-code/EventSourcing.hs
*** https://crdt.tech/
*** [[https://blog.acolyer.org/2014/11/07/highly-available-transactions-virtues-and-limitations/][Highly Available Transactions: Virtues and Limitations]]
*** [[https://medium.com/abrai/demystifying-consistency-and-isolation-for-a-distributed-systems-engineer-64a064c52f6e][Demystifying Consistency and Isolation for a Distributed Systems Engineer]]
*** [[https://ditto.live/][Ditto]]
*** [[https://github.com/TimelyDataflow/timely-dataflow][Timely Dataflow]]
** Misc
*** [[https://github.com/cloudant/sync-android/blob/master/doc/query.md][Cloudant Sync querying]]
*** [[https://gitlab.com/EuAndreh/mediator-old/-/blob/master/resources/Prototype%20of%20Mentat%20Sync.pdf][Syncing Toodle, a.k.a. Mentat Sync Prototype]]
*** [[https://www.youtube.com/watch?v=TSDyxtVbbME][Building Offline First apps with Hoodie]]
Hoodie APIs are a great place to copy from.
*** [[https://lobste.rs/s/m9vkg4/database_i_wish_i_had][lobste.rs on "The database I wish I had"]]
*** [[https://www.reddit.com/r/programming/comments/ijwz5b/the_database_i_wish_i_had/][reddit on "The database I wish I had"]]
*** [[http://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-the-history-youre-looking-for.html][Datomic: this is not the history you're looking for]]
*** [[https://www.youtube.com/playlist?list=PLSE8ODhjZXjagqlf1NxuBQwaMkrHXi-iz][Quarantine Database Tech Talks (2020)]]
*** [[https://www.infoq.com/presentations/Simple-Made-Easy/][Simple Made Easy]]
See 50 minutes mark.
*** RDF and OWL
*** [[https://git.sr.ht/~ireas/merge-rs][merge-rs]]
*** [[https://durmonski.com/self-improvement/the-three-year-rule/][The Three Year Rule: How To Stay Motivated Working On A Long-Term Project]]
*** [[https://blog.eqrion.net/future-directions-for-cbindgen/][Future directions for cbindgen (rust-ffi)]]
*** [[https://developers.cloudflare.com/workers/runtime-apis/kv][Cloudflare Workers KV]]: could this be an storage engine?
*** https://rust-embedded.github.io/book/intro/no-std.html
*** https://news.ycombinator.com/item?id=25167423
** Pilfer from
*** [[https://gist.github.com/refset/a00be06443bc03ccc84a2874af3cdb8a][Initial crux-datoms spike]]
*** [[https://www.opencrux.com/about/bitemporality.html][Crux bitemporality]] and [[https://juxt.pro/blog/value-of-bitemporality][The Value of Bitemporality]]
*** [[https://github.com/juxt/crux/issues/895#issuecomment-636041476][Crux: Hybrid Logical Clock]]
*** [[https://edgedb.com/blog/we-can-do-better-than-sql/][We Can Do Better Than SQL]]
*** [[https://jneem.github.io/merging/][Part 1: Merging and patches]]
*** [[https://github.com/smallhelm/level-fact-base#why-facts][level-fact-base]]
*** [[https://github.com/crubier/Hexastore][Hexastore]]
*** [[https://github.com/ccorcos/datalog-prototype/][datalog-prototype]]
*** [[https://github.com/yjs/yjs][Y.js]]
** Read queue
- https://jaredforsyth.com/posts/local-first-database-gun-js/
- https://jaredforsyth.com/posts/hybrid-logical-clocks/
- https://jaredforsyth.com/posts/local-first-database-rxdb-pouchdb/
- https://jaredforsyth.com/posts/in-search-of-a-local-first-database/
- https://news.ycombinator.com/item?id=24027663
- https://news.ycombinator.com/item?id=19804478
- https://news.ycombinator.com/item?id=21581444
* Scratch
;; MVP: in-memory using Clojure's data structures
;; 1. rolloup a linear log and find a single final, with schema validation
;; 2. rolloup a DAG log and find multiple values
;; 3. query linear log and find a single value
;; 4. query DAG log and find multiple values
;; 5. detect compactible data accretion and conflicts
;; 6. solve conflict with merge transaction
;; 7. when in conflict, allow resolution fns to be provided so it can *sometimes* resolve it
;; 8. during a transaction, allow custom tx-fns to be executed
;; 9. transparent encryption a la git-remote-gcrypt: https://git.spwhitton.name/git-remote-gcrypt/tree/git-remote-gcrypt (could solve item 6 of https://www.inkandswitch.com/local-first.html#privacy)
;; 10. allow the "save all the time" mindset, enabling item 4 of https://www.inkandswitch.com/local-first.html#collaboration
;;
;; On this POC we'll completly ignore:
;; - durability
;; - storage compression
;; - algorithm efficiency
;;
;; etc.
;;
;; Our focus is solely validate the ideas presented above




;; https://docs.datomic.com/on-prem/schema.html#required-schema-attributes
;; More restrictive: requires namespace
;; https://docs.datomic.com/on-prem/schema.html#value-types
;; No composite values (tuples)

;; https://docs.datomic.com/on-prem/schema.html#cardinality

;; https://docs.datomic.com/on-prem/schema.html#operational-schema-attributes

;; With the ::schema spec we can validate the data of the schema declaration,
;; but we still need to validate the actual data when it comes in

see also in documentation and API documentation: like in clojuredocs
examples in documentation and API documentation: like in clojuredocs