Backend Uploading
Admit PushDrop tokens and store file metadata via your overlay’s Topic Manager and Lookup Service.
This page shows the backend changes needed to admit and store the transaction outputs created when a user uploads a file. You'll:
- Rename your Topic Manager and Lookup Service
- Define the UTXO payload fields
- Configure the Topic Manager to admit those fields
- Extend storage to persist all file metadata
- Parse and store admitted outputs in your Lookup Service
- Add documentation for both services
- Verify everything is wired and linked properly
1. Rename and register services
First, install the @bsv/sdk in your backend directory.
1cd backend
2npm install @bsv/sdkAfter that, rename your files under backend/src/topic-managers/ and backend/src/lookup-services/:
1backend/src/
2├── topic-managers/
3│ ├── MarketTopicDocs.md.ts
4│ └── MarketTopicManager.ts
5└── lookup-services/
6 ├── MarketLookupDocs.md.ts
7 ├── MarketStorage.ts
8 └── MarketLookupServiceFactory.tsThen update deployment-info.json to use unique names:
1{
2 "topicManagers": {
3- "tm_template": "./backend/src/topic-managers/TemplateTopicManager.ts"
4+ "tm_demo_market": "./backend/src/topic-managers/MarketTopicManager.ts"
5 },
6 "lookupServices": {
7- "ls_template": {
8+ "ls_demo_market": {
9 "serviceFactory": "./backend/src/lookup-services/MarketLookupServiceFactory.ts",
10 "hydrateWith": "mongo"
11 }
12 },
13 …
14 }Make sure that the name you give it is fully unique to your project! LARS and CARS will not work if service names overlap
2. Locking script payload fields
Your frontend will embed these fields into the PushDrop payload. When decoded by your Topic Manager, expect:
fileUrl– UHRP hash of the encrypted STL file.name– file/listing name.description– store page description.satoshis– sale price.publicKey– uploader's identity key.fileSize– size of STL in bytes.expiryTime– Unix-ms timestamp when file expires.coverUrl– UHRP hash of the (unencrypted) cover image.
3. Configure the Topic Manager
In MarketTopicManager.ts, update identifyAdmissibleOutputs to extract and admit all fields:
1// backend/src/topic-managers/MarketTopicManager.ts
2import { AdmittanceInstructions, TopicManager } from '@bsv/overlay'
3import { Transaction, PublicKey, PushDrop, Utils } from '@bsv/sdk'
4import docs from './MarketTopicDocs.md.js'
5
6export default class MarketTopicManager implements TopicManager {
7 async identifyAdmissibleOutputs(beef: number[], previousCoins: number[]): Promise<AdmittanceInstructions> {
8 const admissibleOutputs: number[] = [];
9
10 try {
11 const decodedTx = Transaction.fromBEEF(beef)
12 const outputs = decodedTx.outputs
13
14 // Try to decode and validate transaction outputs
15 for (const [index, output] of outputs.entries()) {
16 try {
17 const decodedScript = PushDrop.decode(output.lockingScript)
18 const fields = decodedScript.fields
19
20 // checking if all of the fields are structured as expected
21 // UHRP Url
22 if (fields[0].length !== 52) {
23 console.log('Invalid UHRP url length.')
24 continue
25 }
26
27 // Name
28 if (fields[1].length === 0) {
29 console.log('Name field is empty:', fields[1].toString())
30 continue
31 }
32
33 // Description
34 // (Optional validation can be added here if needed)
35
36 // Satoshis
37 const satoshis = parseInt(fields[3].toString(), 10)
38 if (isNaN(satoshis) || satoshis <= 0) {
39 console.log('Invalid satoshis value.')
40 continue
41 }
42
43 // Public Key
44 if (!fields[4] || fields[4].length === 0) {
45 console.log('Public key is missing.')
46 continue
47 }
48 try {
49 console.log('Public Key:', Utils.toUTF8(Utils.toArray(fields[4])))
50 PublicKey.fromString(Utils.toUTF8(Utils.toArray(fields[4])))
51 } catch {
52 console.log('Invalid public key format.')
53 continue
54 }
55
56 // File Size
57 const fileSize = parseInt(fields[5].toString(), 10)
58 if (isNaN(fileSize) || fileSize <= 0) {
59 console.log('Invalid file size.')
60 continue
61 }
62
63 // Expiry Time
64 const expiryTime = parseInt(Utils.toUTF8(Utils.toArray(fields[6])), 10)
65 if (isNaN(expiryTime) || expiryTime <= Date.now()) {
66 console.log('Invalid expiry time:', expiryTime)
67 continue
68 }
69
70 // Cover Image UHRP Url
71 if (fields[7].length !== 52) {
72 console.log('Invalid cover image UHRP url length.')
73 continue
74 }
75
76 admissibleOutputs.push(index)
77 } catch (error) {
78 continue
79 }
80 }
81 if (admissibleOutputs.length === 0) {
82 console.warn('No outputs admitted!')
83 }
84 } catch (error) {
85 console.error('Error identifying admissible outputs:', error)
86 }
87
88 return {
89 outputsToAdmit: admissibleOutputs,
90 coinsToRetain: previousCoins
91 }
92}What's happening:
- Decode PushDrop payload
- Convert fields into typed values
- Ensure the values of each of the eight fields are properly formatted
- Admit the output index of the valid token
4. Extend Lookup Storage
The first thing that you will want to do is to add a new interface for the records that are being stored. This interface will go along with the existing UTXOReference interface in types.ts:
1// backend/src/types.ts
2export interface MarketRecord {
3 fileUrl: string
4 name: string
5 description: string
6 satoshis: number
7 creatorPublicKey: string
8 size: number
9 txid: string
10 outputIndex: number
11 retentionPeriod: number
12 coverUrl: string
13 createdAt: Date // optional field
14}
15
16export interface UTXOReference {
17 txid: string
18 outputIndex: number
19}In TemplateStorage → MarketStorage.ts, update storeRecord signature and add the query isFileUrlInDatabase:
1// backend/src/lookup-services/MarketStorage.ts
2import { Collection, Db } from 'mongodb'
3import { MarketRecord, UTXOReference } from '../types'
4
5export class MarketStorage {
6 private readonly records: Collection<MarketRecord>
7
8 constructor(private readonly db: Db) {
9 this.records = db.collection<MarketRecord>('market_record')
10 }
11
12 async storeRecord(
13 fileUrl: string,
14 name: string,
15 description: string,
16 satoshis: number,
17 creatorPublicKey: string,
18 size: number,
19 txid: string,
20 outputIndex: number,
21 retentionPeriod: number,
22 coverUrl: string
23 ): Promise<void> {
24 // logging when the values are being stored in the
25 console.log("Storing record in MongoDB:", {
26 fileUrl,
27 name,
28 description,
29 satoshis,
30 creatorPublicKey,
31 size,
32 txid,
33 outputIndex,
34 retentionPeriod,
35 coverUrl,
36 createdAt: Date.now()
37 })
38 try {
39 // Insert new record
40 await this.records.insertOne({
41 fileUrl,
42 name,
43 description,
44 satoshis,
45 creatorPublicKey,
46 size,
47 txid,
48 outputIndex,
49 retentionPeriod,
50 coverUrl,
51 createdAt: new Date()
52 })
53 } catch (error) {
54 console.error("Failed to store record:", error)
55 throw error
56 }
57 }
58
59 // … other existing functions (deleteRecord, findAll, etc...) …
60
61 // custom query function
62 async isFileUrlInDatabase(fileUrl: string): Promise<UTXOReference[]> {
63 return await this.records
64 .find({ fileUrl })
65 .project<UTXOReference>({ txid: 1, outputIndex: 1 })
66 .toArray()
67 .then(results => results.map(r => ({
68 txid: r.txid, outputIndex: r.outputIndex
69 })))
70 }
71}The isFileUrlInDatabase query will be used to check if any of the outputs that are stored in the market's database have the provided fileUrl as a field. This will be used on file upload by the key-server to verify the admittance. Note that it does not return a boolean because all queries in the storage class should have the return type UTXOReference.
5. Implement Lookup Service Factory
In MarketLookupServiceFactory.ts, parse all fields and call storeRecord:
1// backend/src/lookup-services/MarketLookupServiceFactory.ts
2import {
3 LookupService, OutputAdmittedByTopic, OutputSpent,
4 LookupQuestion, LookupAnswer, LookupFormula,
5 AdmissionMode, SpendNotificationMode
6} from '@bsv/overlay'
7import { PushDrop, Utils } from '@bsv/sdk'
8import { MarketStorage } from './MarketStorage.js'
9import docs from './MarketLookupDocs.md.js'
10import { Db } from 'mongodb'
11
12
13/**
14 * Template for a Lookup Service that can be modified for your specific use-case.
15 */
16export class MarketLookupService implements LookupService {
17 readonly admissionMode: AdmissionMode = 'locking-script'
18 readonly spendNotificationMode: SpendNotificationMode = 'none'
19
20 constructor(public storage: MarketStorage) { }
21
22 async outputAdmittedByTopic(payload: OutputAdmittedByTopic): Promise<void> {
23 if (payload.mode !== 'locking-script') throw new Error('Invalid mode')
24 const { topic, lockingScript, txid, outputIndex } = payload
25 if (topic !== 'tm_demo_market') throw new Error(`Invalid topic "${topic}" for this service.`)
26 try {
27 const decodedScript = PushDrop.decode(lockingScript)
28 const fields = decodedScript.fields
29
30 const uhrpUrl = Utils.toUTF8(Utils.toArray(fields[0]))
31 const name = Utils.toUTF8(Utils.toArray(fields[1]))
32 const description = Utils.toUTF8(Utils.toArray(fields[2]))
33 const satoshis = Number(Utils.toUTF8(Utils.toArray(fields[3])))
34 const creatorPublicKey = Utils.toUTF8(Utils.toArray(fields[4]))
35 const size = Number(Utils.toUTF8(Utils.toArray(fields[5])))
36 const retentionPeriod = Number(Utils.toUTF8(Utils.toArray(fields[6])))
37 const coverUrl = Utils.toUTF8(Utils.toArray(fields[7]))
38
39 // Store the token fields for future lookup
40 await this.storage.storeRecord(
41 uhrpUrl,
42 name,
43 description,
44 satoshis,
45 creatorPublicKey,
46 size,
47 txid,
48 outputIndex,
49 retentionPeriod,
50 coverUrl
51 )
52 } catch (error) {
53 console.error('Failed to store record', error)
54 return
55 }
56 }
57 // … other existing methods …
58}
59
60// factory
61export default (db: Db) =>
62 new MarketLookupService(new MarketStorage(db))Key steps:
- Decode PushDrop fields array
- Map to typed variables
- Call
storeRecordwith all metadata
Adding the query
In MarketLookupServiceFactory.ts, add this helper function after the class definition to check the type of the query:
1// helper functions
2
3function isUrlCheckQuery(query: any): query is { type: 'urlCheck'; value: { fileUrl: string } } {
4 return (
5 typeof query === 'object' &&
6 query.type === 'urlCheck' &&
7 query.value &&
8 typeof query.value.fileUrl === 'string'
9 );
10}And add the query in the lookup function:
1async lookup(question: LookupQuestion): Promise<LookupAnswer | LookupFormula> {
2 const query = question.query
3
4 // Validate query presence
5 if (!query) {
6 throw new Error('A valid query must be provided!');
7 }
8
9 // Validate service
10 if (question.service !== 'ls_demo_market') {
11 throw new Error('Lookup service not supported!');
12 }
13
14 // Handle specific queries
15 if (query === 'findAll') {
16 return await this.storage.findAll();
17 }
18
19 // Handles the query for a specific fileUrl
20 if (isUrlCheckQuery(query)) {
21 const { fileUrl } = query.value
22 return await this.storage.isFileUrlInDatabase(fileUrl)
23 }
24
25 throw new Error('Unknown query type');
26}6. Adding Documentation
Proper documentation can help users understand what your Overlay Services and will be shown in the UI for the backend server.
6.1 Topic Manager Docs
In backend/src/topic-managers/MarketTopicManager.ts, replace the placeholder shortDescription:
1async getMetaData(): Promise<{
2 name: string
3 shortDescription: string
4 iconURL?: string
5 version?: string
6 informationURL?: string
7}> {
8 return {
9 name: 'Marketplace Topic Manager',
10 shortDescription: 'Admits and filters tokens for the 3D marketplace'
11 }
12}And add to MarketTopicDocs.md.ts with a description of this Topic Manager.
6.2 Lookup Service Docs
In backend/src/lookup-services/MarketLookupServiceFactory.ts:
1async getMetaData(): Promise<{
2 name: string
3 shortDescription: string
4 iconURL?: string
5 version?: string
6 informationURL?: string
7}> {
8 return {
9 name: 'Marketplace Lookup Service',
10 shortDescription: 'Provides search and retrieval of admitted 3D-object listings'
11 }
12}And add to MarketLookupDocs.md.ts with a description of this Lookup Service.
7. Verify Your Overlay Services
Now launch LARS and confirm your new Topic Manager and Lookup Service are loaded:
1## from your project root
2npx larsOpen your browser to http://localhost:8080 In the Topic Managers and Lookup Services menus you should see entries for:
- tm_demo_market → MarketTopicManager
- ls_demo_market → MarketLookupService
If they appear there, your backend wiring is complete! Next up: configuring the key server with middlewares and routes for uploading.
