Backend Uploading

Admit PushDrop tokens and store file metadata via your overlay’s Topic Manager and Lookup Service.

Read as Markdown

This page shows the backend changes needed to admit and store the transaction outputs created when a user uploads a file. You'll:

  1. Rename your Topic Manager and Lookup Service
  2. Define the UTXO payload fields
  3. Configure the Topic Manager to admit those fields
  4. Extend storage to persist all file metadata
  5. Parse and store admitted outputs in your Lookup Service
  6. Add documentation for both services
  7. Verify everything is wired and linked properly

1. Rename and register services

First, install the @bsv/sdk in your backend directory.

1cd backend
2npm install @bsv/sdk
Copied!

After that, rename your files under backend/src/topic-managers/ and backend/src/lookup-services/:

1backend/src/
2├── topic-managers/
3│   ├── MarketTopicDocs.md.ts
4│   └── MarketTopicManager.ts
5└── lookup-services/
6    ├── MarketLookupDocs.md.ts
7    ├── MarketStorage.ts
8    └── MarketLookupServiceFactory.ts
Copied!

Then update deployment-info.json to use unique names:

1{
2   "topicManagers": {
3-    "tm_template": "./backend/src/topic-managers/TemplateTopicManager.ts"
4+    "tm_demo_market": "./backend/src/topic-managers/MarketTopicManager.ts"
5   },
6   "lookupServices": {
7-    "ls_template": {
8+    "ls_demo_market": {
9       "serviceFactory": "./backend/src/lookup-services/MarketLookupServiceFactory.ts",
10       "hydrateWith": "mongo"
11     }
12   },
13   …
14 }
Copied!

Make sure that the name you give it is fully unique to your project! LARS and CARS will not work if service names overlap


2. Locking script payload fields

Your frontend will embed these fields into the PushDrop payload. When decoded by your Topic Manager, expect:

  • fileUrl – UHRP hash of the encrypted STL file.
  • name – file/listing name.
  • description – store page description.
  • satoshis – sale price.
  • publicKey – uploader's identity key.
  • fileSize – size of STL in bytes.
  • expiryTime – Unix-ms timestamp when file expires.
  • coverUrl – UHRP hash of the (unencrypted) cover image.

3. Configure the Topic Manager

In MarketTopicManager.ts, update identifyAdmissibleOutputs to extract and admit all fields:

1// backend/src/topic-managers/MarketTopicManager.ts
2import { AdmittanceInstructions, TopicManager } from '@bsv/overlay'
3import { Transaction, PublicKey, PushDrop, Utils } from '@bsv/sdk'
4import docs from './MarketTopicDocs.md.js'
5
6export default class MarketTopicManager implements TopicManager {
7  async identifyAdmissibleOutputs(beef: number[], previousCoins: number[]): Promise<AdmittanceInstructions> {
8  const admissibleOutputs: number[] = [];
9
10  try {
11    const decodedTx = Transaction.fromBEEF(beef)
12    const outputs = decodedTx.outputs
13
14    // Try to decode and validate transaction outputs
15    for (const [index, output] of outputs.entries()) {
16      try {
17        const decodedScript = PushDrop.decode(output.lockingScript)
18        const fields = decodedScript.fields
19        
20        // checking if all of the fields are structured as expected
21        // UHRP Url
22        if (fields[0].length !== 52) {
23          console.log('Invalid UHRP url length.')
24          continue
25        }
26
27        // Name
28        if (fields[1].length === 0) {
29          console.log('Name field is empty:', fields[1].toString())
30          continue
31        }
32
33        // Description
34        // (Optional validation can be added here if needed)
35
36        // Satoshis
37        const satoshis = parseInt(fields[3].toString(), 10)
38        if (isNaN(satoshis) || satoshis <= 0) {
39          console.log('Invalid satoshis value.')
40          continue
41        }
42
43        // Public Key
44        if (!fields[4] || fields[4].length === 0) {
45          console.log('Public key is missing.')
46          continue
47        }
48        try {
49          console.log('Public Key:', Utils.toUTF8(Utils.toArray(fields[4])))
50          PublicKey.fromString(Utils.toUTF8(Utils.toArray(fields[4])))
51        } catch {
52          console.log('Invalid public key format.')
53          continue
54        }
55
56        // File Size
57        const fileSize = parseInt(fields[5].toString(), 10)
58        if (isNaN(fileSize) || fileSize <= 0) {
59          console.log('Invalid file size.')
60          continue
61        }
62
63        // Expiry Time
64        const expiryTime = parseInt(Utils.toUTF8(Utils.toArray(fields[6])), 10)
65        if (isNaN(expiryTime) || expiryTime <= Date.now()) {
66          console.log('Invalid expiry time:', expiryTime)
67          continue
68        }
69
70        // Cover Image UHRP Url
71        if (fields[7].length !== 52) {
72          console.log('Invalid cover image UHRP url length.')
73          continue
74        }
75
76        admissibleOutputs.push(index)
77      } catch (error) {
78        continue
79      }
80    }
81    if (admissibleOutputs.length === 0) {
82      console.warn('No outputs admitted!')
83    }
84  } catch (error) {
85    console.error('Error identifying admissible outputs:', error)
86  }
87
88  return {
89    outputsToAdmit: admissibleOutputs,
90    coinsToRetain: previousCoins
91  }
92}
Copied!

What's happening:

  1. Decode PushDrop payload
  2. Convert fields into typed values
  3. Ensure the values of each of the eight fields are properly formatted
  4. Admit the output index of the valid token

4. Extend Lookup Storage

The first thing that you will want to do is to add a new interface for the records that are being stored. This interface will go along with the existing UTXOReference interface in types.ts:

1// backend/src/types.ts
2export interface MarketRecord {
3  fileUrl: string
4  name: string
5  description: string
6  satoshis: number
7  creatorPublicKey: string
8  size: number
9  txid: string
10  outputIndex: number
11  retentionPeriod: number
12  coverUrl: string
13  createdAt: Date // optional field
14}
15
16export interface UTXOReference {
17  txid: string
18  outputIndex: number
19}
Copied!

In TemplateStorage → MarketStorage.ts, update storeRecord signature and add the query isFileUrlInDatabase:

1// backend/src/lookup-services/MarketStorage.ts
2import { Collection, Db } from 'mongodb'
3import { MarketRecord, UTXOReference } from '../types'
4
5export class MarketStorage {
6  private readonly records: Collection<MarketRecord>
7
8  constructor(private readonly db: Db) {
9    this.records = db.collection<MarketRecord>('market_record')
10  }
11
12  async storeRecord(
13    fileUrl: string, 
14    name: string, 
15    description: string, 
16    satoshis: number, 
17    creatorPublicKey: string, 
18    size: number, 
19    txid: string, 
20    outputIndex: number, 
21    retentionPeriod: number, 
22    coverUrl: string
23  ): Promise<void> {
24    // logging when the values are being stored in the 
25    console.log("Storing record in MongoDB:", {
26      fileUrl,
27      name,
28      description,
29      satoshis,
30      creatorPublicKey,
31      size,
32      txid,
33      outputIndex,
34      retentionPeriod,
35      coverUrl,
36      createdAt: Date.now()
37    })
38    try {
39      // Insert new record
40      await this.records.insertOne({
41        fileUrl,
42        name,
43        description,
44        satoshis,
45        creatorPublicKey,
46        size,
47        txid,
48        outputIndex,
49        retentionPeriod,
50        coverUrl,
51        createdAt: new Date()
52      })
53    } catch (error) {
54      console.error("Failed to store record:", error)
55      throw error
56    }
57  }
58
59  // … other existing functions (deleteRecord, findAll, etc...) …
60
61  // custom query function
62  async isFileUrlInDatabase(fileUrl: string): Promise<UTXOReference[]> {
63    return await this.records
64      .find({ fileUrl })
65      .project<UTXOReference>({ txid: 1, outputIndex: 1 })
66      .toArray()
67      .then(results => results.map(r => ({
68        txid: r.txid, outputIndex: r.outputIndex
69      })))
70  }
71}
Copied!

The isFileUrlInDatabase query will be used to check if any of the outputs that are stored in the market's database have the provided fileUrl as a field. This will be used on file upload by the key-server to verify the admittance. Note that it does not return a boolean because all queries in the storage class should have the return type UTXOReference.


5. Implement Lookup Service Factory

In MarketLookupServiceFactory.ts, parse all fields and call storeRecord:

1// backend/src/lookup-services/MarketLookupServiceFactory.ts
2import {
3  LookupService, OutputAdmittedByTopic, OutputSpent,
4  LookupQuestion, LookupAnswer, LookupFormula,
5  AdmissionMode, SpendNotificationMode
6} from '@bsv/overlay'
7import { PushDrop, Utils } from '@bsv/sdk'
8import { MarketStorage } from './MarketStorage.js'
9import docs from './MarketLookupDocs.md.js'
10import { Db } from 'mongodb'
11
12
13/**
14 * Template for a Lookup Service that can be modified for your specific use-case.
15 */
16export class MarketLookupService implements LookupService {
17  readonly admissionMode: AdmissionMode = 'locking-script'
18  readonly spendNotificationMode: SpendNotificationMode = 'none'
19
20  constructor(public storage: MarketStorage) { }
21
22  async outputAdmittedByTopic(payload: OutputAdmittedByTopic): Promise<void> {
23    if (payload.mode !== 'locking-script') throw new Error('Invalid mode')
24    const { topic, lockingScript, txid, outputIndex } = payload
25    if (topic !== 'tm_demo_market') throw new Error(`Invalid topic "${topic}" for this service.`)
26    try {
27      const decodedScript = PushDrop.decode(lockingScript)
28      const fields = decodedScript.fields
29
30      const uhrpUrl = Utils.toUTF8(Utils.toArray(fields[0]))
31      const name = Utils.toUTF8(Utils.toArray(fields[1]))
32      const description = Utils.toUTF8(Utils.toArray(fields[2]))
33      const satoshis = Number(Utils.toUTF8(Utils.toArray(fields[3])))
34      const creatorPublicKey = Utils.toUTF8(Utils.toArray(fields[4]))
35      const size = Number(Utils.toUTF8(Utils.toArray(fields[5])))
36      const retentionPeriod = Number(Utils.toUTF8(Utils.toArray(fields[6])))
37      const coverUrl = Utils.toUTF8(Utils.toArray(fields[7]))
38
39      // Store the token fields for future lookup
40      await this.storage.storeRecord(
41        uhrpUrl,
42        name,
43        description,
44        satoshis,
45        creatorPublicKey,
46        size,
47        txid,
48        outputIndex,
49        retentionPeriod,
50        coverUrl
51      )
52    } catch (error) {
53      console.error('Failed to store record', error)
54      return
55    }
56  }
57  // … other existing methods …
58}
59
60// factory
61export default (db: Db) =>
62  new MarketLookupService(new MarketStorage(db))
Copied!

Key steps:

  1. Decode PushDrop fields array
  2. Map to typed variables
  3. Call storeRecord with all metadata

Adding the query

In MarketLookupServiceFactory.ts, add this helper function after the class definition to check the type of the query:

1// helper functions
2
3function isUrlCheckQuery(query: any): query is { type: 'urlCheck'; value: { fileUrl: string } } {
4  return (
5    typeof query === 'object' &&
6    query.type === 'urlCheck' &&
7    query.value &&
8    typeof query.value.fileUrl === 'string'
9  );
10}
Copied!

And add the query in the lookup function:

1async lookup(question: LookupQuestion): Promise<LookupAnswer | LookupFormula> {
2  const query = question.query
3  
4  // Validate query presence
5  if (!query) {
6    throw new Error('A valid query must be provided!');
7  }
8  
9  // Validate service
10  if (question.service !== 'ls_demo_market') {
11    throw new Error('Lookup service not supported!');
12  }
13
14  // Handle specific queries
15  if (query === 'findAll') {
16    return await this.storage.findAll();
17  }  
18
19  // Handles the query for a specific fileUrl
20  if (isUrlCheckQuery(query)) {    
21    const { fileUrl } = query.value
22    return await this.storage.isFileUrlInDatabase(fileUrl)
23  }
24  
25  throw new Error('Unknown query type');
26}
Copied!

6. Adding Documentation

Proper documentation can help users understand what your Overlay Services and will be shown in the UI for the backend server.

6.1 Topic Manager Docs

In backend/src/topic-managers/MarketTopicManager.ts, replace the placeholder shortDescription:

1async getMetaData(): Promise<{
2  name: string
3  shortDescription: string
4  iconURL?: string
5  version?: string
6  informationURL?: string
7}> {
8  return {
9    name: 'Marketplace Topic Manager',
10    shortDescription: 'Admits and filters tokens for the 3D marketplace'
11  }
12}
Copied!

And add to MarketTopicDocs.md.ts with a description of this Topic Manager.

6.2 Lookup Service Docs

In backend/src/lookup-services/MarketLookupServiceFactory.ts:

1async getMetaData(): Promise<{
2  name: string
3  shortDescription: string
4  iconURL?: string
5  version?: string
6  informationURL?: string
7}> {
8  return {
9    name: 'Marketplace Lookup Service',
10    shortDescription: 'Provides search and retrieval of admitted 3D-object listings'
11  }
12}
Copied!

And add to MarketLookupDocs.md.ts with a description of this Lookup Service.


7. Verify Your Overlay Services

Now launch LARS and confirm your new Topic Manager and Lookup Service are loaded:

1## from your project root
2npx lars
Copied!

Open your browser to http://localhost:8080 In the Topic Managers and Lookup Services menus you should see entries for:

  • tm_demo_market → MarketTopicManager
  • ls_demo_market → MarketLookupService

If they appear there, your backend wiring is complete! Next up: configuring the key server with middlewares and routes for uploading.

Was this page helpful?