database reading

This commit is contained in:
micialware committed 2026-09-14 14:32:38 +03:00
1 parent 68c24dcec5
commit fc8455e353
418 files changed
+674410 -1561

No files matched your search

+243
View File
@@ -0,0 +1,243 @@
# SQLite Wasm
SQLite Wasm conveniently wrapped as an ES Module.
## Bug reports
> [!Warning]
>
> This project wraps the code of
> [SQLite Wasm](https://sqlite.org/wasm/doc/trunk/index.md) with _no_ changes,
> apart from added TypeScript types. Please do _not_ file issues or feature
> requests regarding the underlying SQLite Wasm code here. Instead, please
> follow the
> [SQLite bug filing instructions](https://www.sqlite.org/src/wiki?name=Bug+Reports).
> Filing TypeScript type related issues and feature requests is fine.
## Node.js support
> [!Warning]
>
> Node.js is currently only supported for in-memory databases without
> persistence.
## Installation
```bash
npm install @sqlite.org/sqlite-wasm
```
## Usage
There are three ways to use SQLite Wasm:
- [in the main thread with a wrapped worker](#in-a-wrapped-worker-with-opfs-if-available)
(🏆 preferred option)
- [in a worker](#in-a-worker-with-opfs-if-available)
- [in the main thread](#in-the-main-thread-without-opfs)
Only the worker versions allow you to use the origin private file system (OPFS)
storage back-end.
### In a wrapped worker (with OPFS if available):
> [!Warning]
>
> For this to work, you need to set the following headers on your server:
>
> `Cross-Origin-Opener-Policy: same-origin`
>
> `Cross-Origin-Embedder-Policy: require-corp`
```js
import { sqlite3Worker1Promiser } from '@sqlite.org/sqlite-wasm';
const log = console.log;
const error = console.error;
const initializeSQLite = async () => {
try {
log('Loading and initializing SQLite3 module...');
const promiser = await new Promise((resolve) => {
const _promiser = sqlite3Worker1Promiser({
onready: () => resolve(_promiser),
});
});
log('Done initializing. Running demo...');
const configResponse = await promiser('config-get', {});
log('Running SQLite3 version', configResponse.result.version.libVersion);
const openResponse = await promiser('open', {
filename: 'file:mydb.sqlite3?vfs=opfs',
});
const { dbId } = openResponse;
log(
'OPFS is available, created persisted database at',
openResponse.result.filename.replace(/^file:(.*?)\?vfs=opfs$/, '$1'),
);
// Your SQLite code here.
} catch (err) {
if (!(err instanceof Error)) {
err = new Error(err.result.message);
}
error(err.name, err.message);
}
};
initializeSQLite();
```
The `promiser` object above implements the
[Worker1 API](https://sqlite.org/wasm/doc/trunk/api-worker1.md#worker1-methods).
### In a worker (with OPFS if available):
> [!Warning]
>
> For this to work, you need to set the following headers on your server:
>
> `Cross-Origin-Opener-Policy: same-origin`
>
> `Cross-Origin-Embedder-Policy: require-corp`
```js
// In `main.js`.
const worker = new Worker('worker.js', { type: 'module' });
```
```js
// In `worker.js`.
import sqlite3InitModule from '@sqlite.org/sqlite-wasm';
const log = console.log;
const error = console.error;
const start = (sqlite3) => {
log('Running SQLite3 version', sqlite3.version.libVersion);
const db =
'opfs' in sqlite3
? new sqlite3.oo1.OpfsDb('/mydb.sqlite3')
: new sqlite3.oo1.DB('/mydb.sqlite3', 'ct');
log(
'opfs' in sqlite3
? `OPFS is available, created persisted database at ${db.filename}`
: `OPFS is not available, created transient database ${db.filename}`,
);
// Your SQLite code here.
};
const initializeSQLite = async () => {
try {
log('Loading and initializing SQLite3 module...');
const sqlite3 = await sqlite3InitModule({ print: log, printErr: error });
log('Done initializing. Running demo...');
start(sqlite3);
} catch (err) {
error('Initialization error:', err.name, err.message);
}
};
initializeSQLite();
```
The `db` object above implements the
[Object Oriented API #1](https://sqlite.org/wasm/doc/trunk/api-oo1.md).
### In the main thread (without OPFS):
```js
import sqlite3InitModule from '@sqlite.org/sqlite-wasm';
const log = console.log;
const error = console.error;
const start = (sqlite3) => {
log('Running SQLite3 version', sqlite3.version.libVersion);
const db = new sqlite3.oo1.DB('/mydb.sqlite3', 'ct');
// Your SQLite code here.
};
const initializeSQLite = async () => {
try {
log('Loading and initializing SQLite3 module...');
const sqlite3 = await sqlite3InitModule({
print: log,
printErr: error,
});
log('Done initializing. Running demo...');
start(sqlite3);
} catch (err) {
error('Initialization error:', err.name, err.message);
}
};
initializeSQLite();
```
The `db` object above implements the
[Object Oriented API #1](https://sqlite.org/wasm/doc/trunk/api-oo1.md).
## Usage with vite
If you are using [vite](https://vitejs.dev/), you need to add the following
config option in `vite.config.js`:
```js
import { defineConfig } from 'vite';
export default defineConfig({
server: {
headers: {
'Cross-Origin-Opener-Policy': 'same-origin',
'Cross-Origin-Embedder-Policy': 'require-corp',
},
},
optimizeDeps: {
exclude: ['@sqlite.org/sqlite-wasm'],
},
});
```
Check out a
[sample project](https://stackblitz.com/edit/vitejs-vite-ttrbwh?file=main.js)
that shows this in action.
## Demo
See the [demo](https://github.com/sqlite/sqlite-wasm/tree/main/demo) folder for
examples of how to use this in the main thread and in a worker. (Note that the
worker variant requires special HTTP headers, so it can't be hosted on GitHub
Pages.) An example that shows how to use this with vite is available on
[StackBlitz](https://stackblitz.com/edit/vitejs-vite-ttrbwh?file=main.js).
## Projects using this package
See the list of
[npm dependents](https://www.npmjs.com/browse/depended/@sqlite.org/sqlite-wasm)
for this package.
## Deploying a new version
(These steps can only be executed by maintainers.)
1. Update the version number in `package.json` reflecting the current
[SQLite version number](https://sqlite.org/download.html) and add a build
identifier suffix like `-build1`. The complete version number should read
something like `3.41.2-build1`.
1. Run `npm run build` to build the ES Module. This downloads the latest SQLite
Wasm binary and builds the ES Module.
1. Run `npm run deploy` to commit the changes, push to GitHub, and publish the
new version to npm.
## License
Apache 2.0.
## Acknowledgements
This project is based on [SQLite Wasm](https://sqlite.org/wasm), which it
conveniently wraps as an ES Module and publishes to npm as
[`@sqlite.org/sqlite-wasm`](https://www.npmjs.com/package/@sqlite.org/sqlite-wasm).
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env node
import fs from 'fs';
import fetch from 'node-fetch';
import decompress from 'decompress';
async function getSqliteWasmDownloadLink() {
const response = await fetch('https://sqlite.org/download.html');
const html = await response.text();
const sqliteWasmLink =
'https://sqlite.org/' +
html
.replace(
/^.*?<!-- Download product data for scripts to read(.*?)-->.*?$/gms,
'$1',
)
.split(/\n/)
.filter((row) => /sqlite-wasm/.test(row))[0]
.split(/,/)[2];
console.log(`Found SQLite Wasm download link: ${sqliteWasmLink}`);
return sqliteWasmLink;
}
async function downloadAndUnzipSqliteWasm(sqliteWasmDownloadLink) {
if (!sqliteWasmDownloadLink) {
throw new Error('Unable to find SQLite Wasm download link');
}
console.log('Downloading and unzipping SQLite Wasm...');
const response = await fetch(sqliteWasmDownloadLink);
if (!response.ok || response.status !== 200) {
throw new Error(
`Unable to download SQLite Wasm from ${sqliteWasmDownloadLink}`,
);
}
const buffer = await response.arrayBuffer();
fs.writeFileSync('sqlite-wasm.zip', Buffer.from(buffer));
const files = await decompress('sqlite-wasm.zip', 'sqlite-wasm', {
strip: 1,
filter: (file) =>
/jswasm/.test(file.path) && /(\.mjs|\.wasm|\.js)$/.test(file.path),
});
console.log(
`Downloaded and unzipped:\n${files
.map((file) => (/\//.test(file.path) ? '‣ ' + file.path + '\n' : ''))
.join('')}`,
);
fs.rmSync('sqlite-wasm.zip');
}
async function main() {
try {
const sqliteWasmLink = await getSqliteWasmDownloadLink();
await downloadAndUnzipSqliteWasm(sqliteWasmLink);
fs.copyFileSync(
'./node_modules/module-workers-polyfill/module-workers-polyfill.min.js',
'./demo/module-workers-polyfill.min.js',
);
} catch (err) {
console.error(err.name, err.message);
}
}
main();
+8118
View File
File diff suppressed because it is too large. Load diff
+7
View File
@@ -0,0 +1,7 @@
import { default as sqlite3InitModule } from './sqlite-wasm/jswasm/sqlite3-bundler-friendly.mjs';
import './sqlite-wasm/jswasm/sqlite3-worker1-promiser.mjs';
const sqlite3Worker1Promiser = globalThis.sqlite3Worker1Promiser;
export default sqlite3InitModule;
export { sqlite3Worker1Promiser };
+3
View File
@@ -0,0 +1,3 @@
import { default as sqlite3InitModule } from './sqlite-wasm/jswasm/sqlite3-node.mjs';
export default sqlite3InitModule;
+72
View File
@@ -0,0 +1,72 @@
{
"name": "@sqlite.org/sqlite-wasm",
"version": "3.50.4-build1",
"description": "SQLite Wasm conveniently wrapped as an ES Module.",
"keywords": [
"sqlite",
"sqlite3",
"sqlite-wasm",
"sqlite3-wasm",
"webassembly",
"wasm",
"esm",
"opfs",
"origin-private-file-system"
],
"main": "index.mjs",
"node": "node.mjs",
"type": "module",
"files": [
"index.d.ts",
"index.mjs",
"node.mjs",
"sqlite-wasm/"
],
"types": "index.d.ts",
"exports": {
".": {
"types": "./index.d.ts",
"node": "./node.mjs",
"import": "./index.mjs",
"main": "./index.mjs",
"browser": "./index.mjs"
},
"./package.json": "./package.json",
"./sqlite3.wasm": "./sqlite-wasm/jswasm/sqlite3.wasm"
},
"bin": {
"sqlite-wasm": "bin/index.js"
},
"scripts": {
"publint": "npx publint",
"check-types": "tsc",
"clean": "shx rm -rf sqlite-wasm",
"build": "npm run clean && node bin/index.js",
"start": "npx http-server --coop",
"start:node": "cd demo && node node.mjs",
"fix": "npx prettier . --write",
"prepublishOnly": "npm run build && npm run fix && npm run publint && npm run check-types",
"deploy": "npm run prepublishOnly && git add . && git commit -am 'New release' && git push && npm publish --access public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/sqlite/sqlite-wasm.git"
},
"author": "Thomas Steiner (tomac@google.com)",
"license": "Apache-2.0",
"bugs": {
"url": "https://github.com/sqlite/sqlite-wasm/issues"
},
"homepage": "https://github.com/sqlite/sqlite-wasm#readme",
"devDependencies": {
"decompress": "^4.2.1",
"http-server": "github:vapier/http-server",
"module-workers-polyfill": "^0.3.2",
"node-fetch": "^3.3.2",
"prettier": "^3.6.2",
"prettier-plugin-jsdoc": "^1.3.3",
"publint": "^0.3.12",
"shx": "^0.4.0",
"typescript": "^5.9.2"
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,691 @@
/*
2022-09-16
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
A Worker which manages asynchronous OPFS handles on behalf of a
synchronous API which controls it via a combination of Worker
messages, SharedArrayBuffer, and Atomics. It is the asynchronous
counterpart of the API defined in sqlite3-vfs-opfs.js.
Highly indebted to:
https://github.com/rhashimoto/wa-sqlite/blob/master/src/examples/OriginPrivateFileSystemVFS.js
for demonstrating how to use the OPFS APIs.
This file is to be loaded as a Worker. It does not have any direct
access to the sqlite3 JS/WASM bits, so any bits which it needs (most
notably SQLITE_xxx integer codes) have to be imported into it via an
initialization process.
This file represents an implementation detail of a larger piece of
code, and not a public interface. Its details may change at any time
and are not intended to be used by any client-level code.
2022-11-27: Chrome v108 changes some async methods to synchronous, as
documented at:
https://developer.chrome.com/blog/sync-methods-for-accesshandles/
Firefox v111 and Safari 16.4, both released in March 2023, also
include this.
We cannot change to the sync forms at this point without breaking
clients who use Chrome v104-ish or higher. truncate(), getSize(),
flush(), and close() are now (as of v108) synchronous. Calling them
with an "await", as we have to for the async forms, is still legal
with the sync forms but is superfluous. Calling the async forms with
theFunc().then(...) is not compatible with the change to
synchronous, but we do do not use those APIs that way. i.e. we don't
_need_ to change anything for this, but at some point (after Chrome
versions (approximately) 104-107 are extinct) should change our
usage of those methods to remove the "await".
*/
'use strict';
const wPost = (type, ...args) => postMessage({ type, payload: args });
const installAsyncProxy = function () {
const toss = function (...args) {
throw new Error(args.join(' '));
};
if (globalThis.window === globalThis) {
toss(
'This code cannot run from the main thread.',
'Load it as a Worker from a separate Worker.',
);
} else if (!navigator?.storage?.getDirectory) {
toss('This API requires navigator.storage.getDirectory.');
}
const state = Object.create(null);
state.verbose = 1;
const loggers = {
0: console.error.bind(console),
1: console.warn.bind(console),
2: console.log.bind(console),
};
const logImpl = (level, ...args) => {
if (state.verbose > level) loggers[level]('OPFS asyncer:', ...args);
};
const log = (...args) => logImpl(2, ...args);
const warn = (...args) => logImpl(1, ...args);
const error = (...args) => logImpl(0, ...args);
const __openFiles = Object.create(null);
const __implicitLocks = new Set();
const getResolvedPath = function (filename, splitIt) {
const p = new URL(filename, 'file://irrelevant').pathname;
return splitIt ? p.split('/').filter((v) => !!v) : p;
};
const getDirForFilename = async function f(absFilename, createDirs = false) {
const path = getResolvedPath(absFilename, true);
const filename = path.pop();
let dh = state.rootDir;
for (const dirName of path) {
if (dirName) {
dh = await dh.getDirectoryHandle(dirName, { create: !!createDirs });
}
}
return [dh, filename];
};
const closeSyncHandle = async (fh) => {
if (fh.syncHandle) {
log('Closing sync handle for', fh.filenameAbs);
const h = fh.syncHandle;
delete fh.syncHandle;
delete fh.xLock;
__implicitLocks.delete(fh.fid);
return h.close();
}
};
const closeSyncHandleNoThrow = async (fh) => {
try {
await closeSyncHandle(fh);
} catch (e) {
warn('closeSyncHandleNoThrow() ignoring:', e, fh);
}
};
const releaseImplicitLocks = async () => {
if (__implicitLocks.size) {
for (const fid of __implicitLocks) {
const fh = __openFiles[fid];
await closeSyncHandleNoThrow(fh);
log('Auto-unlocked', fid, fh.filenameAbs);
}
}
};
const releaseImplicitLock = async (fh) => {
if (fh.releaseImplicitLocks && __implicitLocks.has(fh.fid)) {
return closeSyncHandleNoThrow(fh);
}
};
class GetSyncHandleError extends Error {
constructor(errorObject, ...msg) {
super(
[...msg, ': ' + errorObject.name + ':', errorObject.message].join(' '),
{
cause: errorObject,
},
);
this.name = 'GetSyncHandleError';
}
}
GetSyncHandleError.convertRc = (e, rc) => {
if (e instanceof GetSyncHandleError) {
if (
e.cause.name === 'NoModificationAllowedError' ||
(e.cause.name === 'DOMException' &&
0 === e.cause.message.indexOf('Access Handles cannot'))
) {
return state.sq3Codes.SQLITE_BUSY;
} else if ('NotFoundError' === e.cause.name) {
return state.sq3Codes.SQLITE_CANTOPEN;
}
} else if ('NotFoundError' === e?.name) {
return state.sq3Codes.SQLITE_CANTOPEN;
}
return rc;
};
const getSyncHandle = async (fh, opName) => {
if (!fh.syncHandle) {
const t = performance.now();
log('Acquiring sync handle for', fh.filenameAbs);
const maxTries = 6,
msBase = state.asyncIdleWaitTime * 2;
let i = 1,
ms = msBase;
for (; true; ms = msBase * ++i) {
try {
fh.syncHandle = await fh.fileHandle.createSyncAccessHandle();
break;
} catch (e) {
if (i === maxTries) {
throw new GetSyncHandleError(
e,
'Error getting sync handle for',
opName + '().',
maxTries,
'attempts failed.',
fh.filenameAbs,
);
}
warn(
'Error getting sync handle for',
opName + '(). Waiting',
ms,
'ms and trying again.',
fh.filenameAbs,
e,
);
Atomics.wait(state.sabOPView, state.opIds.retry, 0, ms);
}
}
log(
'Got',
opName + '() sync handle for',
fh.filenameAbs,
'in',
performance.now() - t,
'ms',
);
if (!fh.xLock) {
__implicitLocks.add(fh.fid);
log(
'Acquired implicit lock for',
opName + '()',
fh.fid,
fh.filenameAbs,
);
}
}
return fh.syncHandle;
};
const storeAndNotify = (opName, value) => {
log(opName + '() => notify(', value, ')');
Atomics.store(state.sabOPView, state.opIds.rc, value);
Atomics.notify(state.sabOPView, state.opIds.rc);
};
const affirmNotRO = function (opName, fh) {
if (fh.readOnly) toss(opName + '(): File is read-only: ' + fh.filenameAbs);
};
let flagAsyncShutdown = false;
const vfsAsyncImpls = {
'opfs-async-shutdown': async () => {
flagAsyncShutdown = true;
storeAndNotify('opfs-async-shutdown', 0);
},
mkdir: async (dirname) => {
let rc = 0;
try {
await getDirForFilename(dirname + '/filepart', true);
} catch (e) {
state.s11n.storeException(2, e);
rc = state.sq3Codes.SQLITE_IOERR;
}
storeAndNotify('mkdir', rc);
},
xAccess: async (filename) => {
let rc = 0;
try {
const [dh, fn] = await getDirForFilename(filename);
await dh.getFileHandle(fn);
} catch (e) {
state.s11n.storeException(2, e);
rc = state.sq3Codes.SQLITE_IOERR;
}
storeAndNotify('xAccess', rc);
},
xClose: async function (fid) {
const opName = 'xClose';
__implicitLocks.delete(fid);
const fh = __openFiles[fid];
let rc = 0;
if (fh) {
delete __openFiles[fid];
await closeSyncHandle(fh);
if (fh.deleteOnClose) {
try {
await fh.dirHandle.removeEntry(fh.filenamePart);
} catch (e) {
warn('Ignoring dirHandle.removeEntry() failure of', fh, e);
}
}
} else {
state.s11n.serialize();
rc = state.sq3Codes.SQLITE_NOTFOUND;
}
storeAndNotify(opName, rc);
},
xDelete: async function (...args) {
const rc = await vfsAsyncImpls.xDeleteNoWait(...args);
storeAndNotify('xDelete', rc);
},
xDeleteNoWait: async function (filename, syncDir = 0, recursive = false) {
let rc = 0;
try {
while (filename) {
const [hDir, filenamePart] = await getDirForFilename(filename, false);
if (!filenamePart) break;
await hDir.removeEntry(filenamePart, { recursive });
if (0x1234 !== syncDir) break;
recursive = false;
filename = getResolvedPath(filename, true);
filename.pop();
filename = filename.join('/');
}
} catch (e) {
state.s11n.storeException(2, e);
rc = state.sq3Codes.SQLITE_IOERR_DELETE;
}
return rc;
},
xFileSize: async function (fid) {
const fh = __openFiles[fid];
let rc = 0;
try {
const sz = await (await getSyncHandle(fh, 'xFileSize')).getSize();
state.s11n.serialize(Number(sz));
} catch (e) {
state.s11n.storeException(1, e);
rc = GetSyncHandleError.convertRc(e, state.sq3Codes.SQLITE_IOERR);
}
await releaseImplicitLock(fh);
storeAndNotify('xFileSize', rc);
},
xLock: async function (fid, lockType) {
const fh = __openFiles[fid];
let rc = 0;
const oldLockType = fh.xLock;
fh.xLock = lockType;
if (!fh.syncHandle) {
try {
await getSyncHandle(fh, 'xLock');
__implicitLocks.delete(fid);
} catch (e) {
state.s11n.storeException(1, e);
rc = GetSyncHandleError.convertRc(
e,
state.sq3Codes.SQLITE_IOERR_LOCK,
);
fh.xLock = oldLockType;
}
}
storeAndNotify('xLock', rc);
},
xOpen: async function (fid, filename, flags, opfsFlags) {
const opName = 'xOpen';
const create = state.sq3Codes.SQLITE_OPEN_CREATE & flags;
try {
let hDir, filenamePart;
try {
[hDir, filenamePart] = await getDirForFilename(filename, !!create);
} catch (e) {
state.s11n.storeException(1, e);
storeAndNotify(opName, state.sq3Codes.SQLITE_NOTFOUND);
return;
}
if (state.opfsFlags.OPFS_UNLINK_BEFORE_OPEN & opfsFlags) {
try {
await hDir.removeEntry(filenamePart);
} catch (e) {}
}
const hFile = await hDir.getFileHandle(filenamePart, { create });
const fh = Object.assign(Object.create(null), {
fid: fid,
filenameAbs: filename,
filenamePart: filenamePart,
dirHandle: hDir,
fileHandle: hFile,
sabView: state.sabFileBufView,
readOnly: !create && !!(state.sq3Codes.SQLITE_OPEN_READONLY & flags),
deleteOnClose: !!(state.sq3Codes.SQLITE_OPEN_DELETEONCLOSE & flags),
});
fh.releaseImplicitLocks =
opfsFlags & state.opfsFlags.OPFS_UNLOCK_ASAP ||
state.opfsFlags.defaultUnlockAsap;
__openFiles[fid] = fh;
storeAndNotify(opName, 0);
} catch (e) {
error(opName, e);
state.s11n.storeException(1, e);
storeAndNotify(opName, state.sq3Codes.SQLITE_IOERR);
}
},
xRead: async function (fid, n, offset64) {
let rc = 0,
nRead;
const fh = __openFiles[fid];
try {
nRead = (await getSyncHandle(fh, 'xRead')).read(
fh.sabView.subarray(0, n),
{ at: Number(offset64) },
);
if (nRead < n) {
fh.sabView.fill(0, nRead, n);
rc = state.sq3Codes.SQLITE_IOERR_SHORT_READ;
}
} catch (e) {
error('xRead() failed', e, fh);
state.s11n.storeException(1, e);
rc = GetSyncHandleError.convertRc(e, state.sq3Codes.SQLITE_IOERR_READ);
}
await releaseImplicitLock(fh);
storeAndNotify('xRead', rc);
},
xSync: async function (fid, flags) {
const fh = __openFiles[fid];
let rc = 0;
if (!fh.readOnly && fh.syncHandle) {
try {
await fh.syncHandle.flush();
} catch (e) {
state.s11n.storeException(2, e);
rc = state.sq3Codes.SQLITE_IOERR_FSYNC;
}
}
storeAndNotify('xSync', rc);
},
xTruncate: async function (fid, size) {
let rc = 0;
const fh = __openFiles[fid];
try {
affirmNotRO('xTruncate', fh);
await (await getSyncHandle(fh, 'xTruncate')).truncate(size);
} catch (e) {
error('xTruncate():', e, fh);
state.s11n.storeException(2, e);
rc = GetSyncHandleError.convertRc(
e,
state.sq3Codes.SQLITE_IOERR_TRUNCATE,
);
}
await releaseImplicitLock(fh);
storeAndNotify('xTruncate', rc);
},
xUnlock: async function (fid, lockType) {
let rc = 0;
const fh = __openFiles[fid];
if (fh.syncHandle && state.sq3Codes.SQLITE_LOCK_NONE === lockType) {
try {
await closeSyncHandle(fh);
} catch (e) {
state.s11n.storeException(1, e);
rc = state.sq3Codes.SQLITE_IOERR_UNLOCK;
}
}
storeAndNotify('xUnlock', rc);
},
xWrite: async function (fid, n, offset64) {
let rc;
const fh = __openFiles[fid];
try {
affirmNotRO('xWrite', fh);
rc =
n ===
(await getSyncHandle(fh, 'xWrite')).write(fh.sabView.subarray(0, n), {
at: Number(offset64),
})
? 0
: state.sq3Codes.SQLITE_IOERR_WRITE;
} catch (e) {
error('xWrite():', e, fh);
state.s11n.storeException(1, e);
rc = GetSyncHandleError.convertRc(e, state.sq3Codes.SQLITE_IOERR_WRITE);
}
await releaseImplicitLock(fh);
storeAndNotify('xWrite', rc);
},
};
const initS11n = () => {
if (state.s11n) return state.s11n;
const textDecoder = new TextDecoder(),
textEncoder = new TextEncoder('utf-8'),
viewU8 = new Uint8Array(
state.sabIO,
state.sabS11nOffset,
state.sabS11nSize,
),
viewDV = new DataView(
state.sabIO,
state.sabS11nOffset,
state.sabS11nSize,
);
state.s11n = Object.create(null);
const TypeIds = Object.create(null);
TypeIds.number = {
id: 1,
size: 8,
getter: 'getFloat64',
setter: 'setFloat64',
};
TypeIds.bigint = {
id: 2,
size: 8,
getter: 'getBigInt64',
setter: 'setBigInt64',
};
TypeIds.boolean = {
id: 3,
size: 4,
getter: 'getInt32',
setter: 'setInt32',
};
TypeIds.string = { id: 4 };
const getTypeId = (v) =>
TypeIds[typeof v] ||
toss('Maintenance required: this value type cannot be serialized.', v);
const getTypeIdById = (tid) => {
switch (tid) {
case TypeIds.number.id:
return TypeIds.number;
case TypeIds.bigint.id:
return TypeIds.bigint;
case TypeIds.boolean.id:
return TypeIds.boolean;
case TypeIds.string.id:
return TypeIds.string;
default:
toss('Invalid type ID:', tid);
}
};
state.s11n.deserialize = function (clear = false) {
const argc = viewU8[0];
const rc = argc ? [] : null;
if (argc) {
const typeIds = [];
let offset = 1,
i,
n,
v;
for (i = 0; i < argc; ++i, ++offset) {
typeIds.push(getTypeIdById(viewU8[offset]));
}
for (i = 0; i < argc; ++i) {
const t = typeIds[i];
if (t.getter) {
v = viewDV[t.getter](offset, state.littleEndian);
offset += t.size;
} else {
n = viewDV.getInt32(offset, state.littleEndian);
offset += 4;
v = textDecoder.decode(viewU8.slice(offset, offset + n));
offset += n;
}
rc.push(v);
}
}
if (clear) viewU8[0] = 0;
return rc;
};
state.s11n.serialize = function (...args) {
if (args.length) {
const typeIds = [];
let i = 0,
offset = 1;
viewU8[0] = args.length & 0xff;
for (; i < args.length; ++i, ++offset) {
typeIds.push(getTypeId(args[i]));
viewU8[offset] = typeIds[i].id;
}
for (i = 0; i < args.length; ++i) {
const t = typeIds[i];
if (t.setter) {
viewDV[t.setter](offset, args[i], state.littleEndian);
offset += t.size;
} else {
const s = textEncoder.encode(args[i]);
viewDV.setInt32(offset, s.byteLength, state.littleEndian);
offset += 4;
viewU8.set(s, offset);
offset += s.byteLength;
}
}
} else {
viewU8[0] = 0;
}
};
state.s11n.storeException = state.asyncS11nExceptions
? (priority, e) => {
if (priority <= state.asyncS11nExceptions) {
state.s11n.serialize([e.name, ': ', e.message].join(''));
}
}
: () => {};
return state.s11n;
};
const waitLoop = async function f() {
const opHandlers = Object.create(null);
for (let k of Object.keys(state.opIds)) {
const vi = vfsAsyncImpls[k];
if (!vi) continue;
const o = Object.create(null);
opHandlers[state.opIds[k]] = o;
o.key = k;
o.f = vi;
}
while (!flagAsyncShutdown) {
try {
if (
'not-equal' !==
Atomics.wait(
state.sabOPView,
state.opIds.whichOp,
0,
state.asyncIdleWaitTime,
)
) {
await releaseImplicitLocks();
continue;
}
const opId = Atomics.load(state.sabOPView, state.opIds.whichOp);
Atomics.store(state.sabOPView, state.opIds.whichOp, 0);
const hnd =
opHandlers[opId] ?? toss('No waitLoop handler for whichOp #', opId);
const args = state.s11n.deserialize(true) || [];
if (hnd.f) await hnd.f(...args);
else error('Missing callback for opId', opId);
} catch (e) {
error('in waitLoop():', e);
}
}
};
navigator.storage
.getDirectory()
.then(function (d) {
state.rootDir = d;
globalThis.onmessage = function ({ data }) {
switch (data.type) {
case 'opfs-async-init': {
const opt = data.args;
for (const k in opt) state[k] = opt[k];
state.verbose = opt.verbose ?? 1;
state.sabOPView = new Int32Array(state.sabOP);
state.sabFileBufView = new Uint8Array(
state.sabIO,
0,
state.fileBufferSize,
);
state.sabS11nView = new Uint8Array(
state.sabIO,
state.sabS11nOffset,
state.sabS11nSize,
);
Object.keys(vfsAsyncImpls).forEach((k) => {
if (!Number.isFinite(state.opIds[k])) {
toss('Maintenance required: missing state.opIds[', k, ']');
}
});
initS11n();
log('init state', state);
wPost('opfs-async-inited');
waitLoop();
break;
}
case 'opfs-async-restart':
if (flagAsyncShutdown) {
warn(
'Restarting after opfs-async-shutdown. Might or might not work.',
);
flagAsyncShutdown = false;
waitLoop();
}
break;
}
};
wPost('opfs-async-loaded');
})
.catch((e) => error('error initializing OPFS asyncer:', e));
};
if (!globalThis.SharedArrayBuffer) {
wPost(
'opfs-unavailable',
'Missing SharedArrayBuffer API.',
'The server must emit the COOP/COEP response headers to enable that.',
);
} else if (!globalThis.Atomics) {
wPost(
'opfs-unavailable',
'Missing Atomics API.',
'The server must emit the COOP/COEP response headers to enable that.',
);
} else if (
!globalThis.FileSystemHandle ||
!globalThis.FileSystemDirectoryHandle ||
!globalThis.FileSystemFileHandle ||
!globalThis.FileSystemFileHandle.prototype.createSyncAccessHandle ||
!navigator?.storage?.getDirectory
) {
wPost('opfs-unavailable', 'Missing required OPFS APIs.');
} else {
installAsyncProxy();
}
@@ -0,0 +1,35 @@
/*
2022-05-23
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This is a JS Worker file for the main sqlite3 api. It loads
sqlite3.js, initializes the module, and postMessage()'s a message
after the module is initialized:
{type: 'sqlite3-api', result: 'worker1-ready'}
This seemingly superfluous level of indirection is necessary when
loading sqlite3.js via a Worker. Instantiating a worker with new
Worker("sqlite.js") will not (cannot) call sqlite3InitModule() to
initialize the module due to a timing/order-of-operations conflict
(and that symbol is not exported in a way that a Worker loading it
that way can see it). Thus JS code wanting to load the sqlite3
Worker-specific API needs to pass _this_ file (or equivalent) to the
Worker constructor and then listen for an event in the form shown
above in order to know when the module has completed initialization.
This file accepts a URL arguments to adjust how it loads sqlite3.js:
- `sqlite3.dir`, if set, treats the given directory name as the
directory from which `sqlite3.js` will be loaded.
*/
import { default as sqlite3InitModule } from './sqlite3-bundler-friendly.mjs';
sqlite3InitModule().then((sqlite3) => sqlite3.initWorker1API());
@@ -0,0 +1,193 @@
/*
2022-08-24
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This file implements a Promise-based proxy for the sqlite3 Worker
API #1. It is intended to be included either from the main thread or
a Worker, but only if (A) the environment supports nested Workers
and (B) it's _not_ a Worker which loads the sqlite3 WASM/JS
module. This file's features will load that module and provide a
slightly simpler client-side interface than the slightly-lower-level
Worker API does.
This script necessarily exposes one global symbol, but clients may
freely `delete` that symbol after calling it.
*/
'use strict';
globalThis.sqlite3Worker1Promiser = function callee(
config = callee.defaultConfig,
) {
if (1 === arguments.length && 'function' === typeof arguments[0]) {
const f = config;
config = Object.assign(Object.create(null), callee.defaultConfig);
config.onready = f;
} else {
config = Object.assign(Object.create(null), callee.defaultConfig, config);
}
const handlerMap = Object.create(null);
const noop = function () {};
const err = config.onerror || noop;
const debug = config.debug || noop;
const idTypeMap = config.generateMessageId ? undefined : Object.create(null);
const genMsgId =
config.generateMessageId ||
function (msg) {
return (
msg.type + '#' + (idTypeMap[msg.type] = (idTypeMap[msg.type] || 0) + 1)
);
};
const toss = (...args) => {
throw new Error(args.join(' '));
};
if (!config.worker) config.worker = callee.defaultConfig.worker;
if ('function' === typeof config.worker) config.worker = config.worker();
let dbId;
let promiserFunc;
config.worker.onmessage = function (ev) {
ev = ev.data;
debug('worker1.onmessage', ev);
let msgHandler = handlerMap[ev.messageId];
if (!msgHandler) {
if (ev && 'sqlite3-api' === ev.type && 'worker1-ready' === ev.result) {
if (config.onready) config.onready(promiserFunc);
return;
}
msgHandler = handlerMap[ev.type];
if (msgHandler && msgHandler.onrow) {
msgHandler.onrow(ev);
return;
}
if (config.onunhandled) config.onunhandled(arguments[0]);
else err('sqlite3Worker1Promiser() unhandled worker message:', ev);
return;
}
delete handlerMap[ev.messageId];
switch (ev.type) {
case 'error':
msgHandler.reject(ev);
return;
case 'open':
if (!dbId) dbId = ev.dbId;
break;
case 'close':
if (ev.dbId === dbId) dbId = undefined;
break;
default:
break;
}
try {
msgHandler.resolve(ev);
} catch (e) {
msgHandler.reject(e);
}
};
return (promiserFunc = function () {
let msg;
if (1 === arguments.length) {
msg = arguments[0];
} else if (2 === arguments.length) {
msg = Object.create(null);
msg.type = arguments[0];
msg.args = arguments[1];
msg.dbId = msg.args.dbId;
} else {
toss('Invalid arguments for sqlite3Worker1Promiser()-created factory.');
}
if (!msg.dbId && msg.type !== 'open') msg.dbId = dbId;
msg.messageId = genMsgId(msg);
msg.departureTime = performance.now();
const proxy = Object.create(null);
proxy.message = msg;
let rowCallbackId;
if ('exec' === msg.type && msg.args) {
if ('function' === typeof msg.args.callback) {
rowCallbackId = msg.messageId + ':row';
proxy.onrow = msg.args.callback;
msg.args.callback = rowCallbackId;
handlerMap[rowCallbackId] = proxy;
} else if ('string' === typeof msg.args.callback) {
toss(
'exec callback may not be a string when using the Promise interface.',
);
}
}
let p = new Promise(function (resolve, reject) {
proxy.resolve = resolve;
proxy.reject = reject;
handlerMap[msg.messageId] = proxy;
debug(
'Posting',
msg.type,
'message to Worker dbId=' + (dbId || 'default') + ':',
msg,
);
config.worker.postMessage(msg);
});
if (rowCallbackId) p = p.finally(() => delete handlerMap[rowCallbackId]);
return p;
});
};
globalThis.sqlite3Worker1Promiser.defaultConfig = {
worker: function () {
let theJs = 'sqlite3-worker1.js';
if (this.currentScript) {
const src = this.currentScript.src.split('/');
src.pop();
theJs = src.join('/') + '/' + theJs;
} else if (globalThis.location) {
const urlParams = new URL(globalThis.location.href).searchParams;
if (urlParams.has('sqlite3.dir')) {
theJs = urlParams.get('sqlite3.dir') + '/' + theJs;
}
}
return new Worker(theJs + globalThis.location.search);
}.bind({
currentScript: globalThis?.document?.currentScript,
}),
onerror: (...args) => console.error('worker1 promiser error', ...args),
};
sqlite3Worker1Promiser.v2 = function (config) {
let oldFunc;
if ('function' == typeof config) {
oldFunc = config;
config = {};
} else if ('function' === typeof config?.onready) {
oldFunc = config.onready;
delete config.onready;
}
const promiseProxy = Object.create(null);
config = Object.assign(config || Object.create(null), {
onready: async function (func) {
try {
if (oldFunc) await oldFunc(func);
promiseProxy.resolve(func);
} catch (e) {
promiseProxy.reject(e);
}
},
});
const p = new Promise(function (resolve, reject) {
promiseProxy.resolve = resolve;
promiseProxy.reject = reject;
});
try {
this.original(config);
} catch (e) {
promiseProxy.reject(e);
}
return p;
}.bind({
original: sqlite3Worker1Promiser,
});
@@ -0,0 +1,187 @@
/*
2022-08-24
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This file implements a Promise-based proxy for the sqlite3 Worker
API #1. It is intended to be included either from the main thread or
a Worker, but only if (A) the environment supports nested Workers
and (B) it's _not_ a Worker which loads the sqlite3 WASM/JS
module. This file's features will load that module and provide a
slightly simpler client-side interface than the slightly-lower-level
Worker API does.
This script necessarily exposes one global symbol, but clients may
freely `delete` that symbol after calling it.
*/
'use strict';
globalThis.sqlite3Worker1Promiser = function callee(
config = callee.defaultConfig,
) {
if (1 === arguments.length && 'function' === typeof arguments[0]) {
const f = config;
config = Object.assign(Object.create(null), callee.defaultConfig);
config.onready = f;
} else {
config = Object.assign(Object.create(null), callee.defaultConfig, config);
}
const handlerMap = Object.create(null);
const noop = function () {};
const err = config.onerror || noop;
const debug = config.debug || noop;
const idTypeMap = config.generateMessageId ? undefined : Object.create(null);
const genMsgId =
config.generateMessageId ||
function (msg) {
return (
msg.type + '#' + (idTypeMap[msg.type] = (idTypeMap[msg.type] || 0) + 1)
);
};
const toss = (...args) => {
throw new Error(args.join(' '));
};
if (!config.worker) config.worker = callee.defaultConfig.worker;
if ('function' === typeof config.worker) config.worker = config.worker();
let dbId;
let promiserFunc;
config.worker.onmessage = function (ev) {
ev = ev.data;
debug('worker1.onmessage', ev);
let msgHandler = handlerMap[ev.messageId];
if (!msgHandler) {
if (ev && 'sqlite3-api' === ev.type && 'worker1-ready' === ev.result) {
if (config.onready) config.onready(promiserFunc);
return;
}
msgHandler = handlerMap[ev.type];
if (msgHandler && msgHandler.onrow) {
msgHandler.onrow(ev);
return;
}
if (config.onunhandled) config.onunhandled(arguments[0]);
else err('sqlite3Worker1Promiser() unhandled worker message:', ev);
return;
}
delete handlerMap[ev.messageId];
switch (ev.type) {
case 'error':
msgHandler.reject(ev);
return;
case 'open':
if (!dbId) dbId = ev.dbId;
break;
case 'close':
if (ev.dbId === dbId) dbId = undefined;
break;
default:
break;
}
try {
msgHandler.resolve(ev);
} catch (e) {
msgHandler.reject(e);
}
};
return (promiserFunc = function () {
let msg;
if (1 === arguments.length) {
msg = arguments[0];
} else if (2 === arguments.length) {
msg = Object.create(null);
msg.type = arguments[0];
msg.args = arguments[1];
msg.dbId = msg.args.dbId;
} else {
toss('Invalid arguments for sqlite3Worker1Promiser()-created factory.');
}
if (!msg.dbId && msg.type !== 'open') msg.dbId = dbId;
msg.messageId = genMsgId(msg);
msg.departureTime = performance.now();
const proxy = Object.create(null);
proxy.message = msg;
let rowCallbackId;
if ('exec' === msg.type && msg.args) {
if ('function' === typeof msg.args.callback) {
rowCallbackId = msg.messageId + ':row';
proxy.onrow = msg.args.callback;
msg.args.callback = rowCallbackId;
handlerMap[rowCallbackId] = proxy;
} else if ('string' === typeof msg.args.callback) {
toss(
'exec callback may not be a string when using the Promise interface.',
);
}
}
let p = new Promise(function (resolve, reject) {
proxy.resolve = resolve;
proxy.reject = reject;
handlerMap[msg.messageId] = proxy;
debug(
'Posting',
msg.type,
'message to Worker dbId=' + (dbId || 'default') + ':',
msg,
);
config.worker.postMessage(msg);
});
if (rowCallbackId) p = p.finally(() => delete handlerMap[rowCallbackId]);
return p;
});
};
globalThis.sqlite3Worker1Promiser.defaultConfig = {
worker: function () {
return new Worker(
new URL('sqlite3-worker1-bundler-friendly.mjs', import.meta.url),
{
type: 'module',
},
);
},
onerror: (...args) => console.error('worker1 promiser error', ...args),
};
sqlite3Worker1Promiser.v2 = function (config) {
let oldFunc;
if ('function' == typeof config) {
oldFunc = config;
config = {};
} else if ('function' === typeof config?.onready) {
oldFunc = config.onready;
delete config.onready;
}
const promiseProxy = Object.create(null);
config = Object.assign(config || Object.create(null), {
onready: async function (func) {
try {
if (oldFunc) await oldFunc(func);
promiseProxy.resolve(func);
} catch (e) {
promiseProxy.reject(e);
}
},
});
const p = new Promise(function (resolve, reject) {
promiseProxy.resolve = resolve;
promiseProxy.reject = reject;
});
try {
this.original(config);
} catch (e) {
promiseProxy.reject(e);
}
return p;
}.bind({
original: sqlite3Worker1Promiser,
});
export default sqlite3Worker1Promiser.v2;
@@ -0,0 +1,46 @@
/*
2022-05-23
The author disclaims copyright to this source code. In place of a
legal notice, here is a blessing:
* May you do good and not evil.
* May you find forgiveness for yourself and forgive others.
* May you share freely, never taking more than you give.
***********************************************************************
This is a JS Worker file for the main sqlite3 api. It loads
sqlite3.js, initializes the module, and postMessage()'s a message
after the module is initialized:
{type: 'sqlite3-api', result: 'worker1-ready'}
This seemingly superfluous level of indirection is necessary when
loading sqlite3.js via a Worker. Instantiating a worker with new
Worker("sqlite.js") will not (cannot) call sqlite3InitModule() to
initialize the module due to a timing/order-of-operations conflict
(and that symbol is not exported in a way that a Worker loading it
that way can see it). Thus JS code wanting to load the sqlite3
Worker-specific API needs to pass _this_ file (or equivalent) to the
Worker constructor and then listen for an event in the form shown
above in order to know when the module has completed initialization.
This file accepts a URL arguments to adjust how it loads sqlite3.js:
- `sqlite3.dir`, if set, treats the given directory name as the
directory from which `sqlite3.js` will be loaded.
*/
'use strict';
{
const urlParams = globalThis.location
? new URL(globalThis.location.href).searchParams
: new URLSearchParams();
let theJs = 'sqlite3.js';
if (urlParams.has('sqlite3.dir')) {
theJs = urlParams.get('sqlite3.dir') + '/' + theJs;
}
importScripts(theJs);
}
sqlite3InitModule().then((sqlite3) => sqlite3.initWorker1API());
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Binary file not shown.