# Data source research notes

Derived by cloning and reading the [Mids Reborn](https://github.com/LoadedCamel/MidsReborn) source (GPL-3.0) directly, not from documentation (there isn't much).

## 1. Build import: the "MBD" data-chunk format

Source: `MidsReborn/Core/BuildFile/CharacterBuildData.cs`, `MidsReborn/Core/Utils/Compression.cs`, `MidsReborn/Core/Utils/DataClassifier.cs`, `MidsReborn/Core/Utils/ModernZlib.cs`.

**Wire format** (what a player pastes from Mids' "Export build"):

```
|MBD;<uncompressedSize>;<compressedSize>;<encodedSize>;BASE64;|
<base64 body, wrapped at 67 chars/line, each line bookended with "|">
```

Classification regex used by Mids itself: `^\|MBD;[0-9]+;[0-9]+;[0-9]+;BASE64;\|`.

**Decode steps:**
1. Split off the header line (starts with `|MBD;`).
2. For the body: strip the leading/trailing `|` on each line and join the lines back into one base64 string (reverses `ModernZlib.BreakString`).
3. Base64-decode.
4. **Brotli-decompress** (Mids uses `System.IO.Compression.BrotliStream` — the compression is plain Brotli, nothing custom).
5. UTF-8 decode → JSON text.
6. Parse JSON into the schema below.

**JSON schema** (`CharacterBuildData` + nested types, all in `MidsReborn/Core/BuildFile/DataModels/`):

```
CharacterBuildData {
  BuiltWith: { App, Version, Database, DatabaseVersion }   // metadata only
  Level: string
  Class: string          // archetype full class name (UID-style)
  Origin: string
  Alignment: string
  Name: string
  Comment: string?
  PowerSets: string[]     // full names of chosen powersets (primary/secondary/pools/epic)
  LastPower: int
  PowerEntries: PowerData[]
}

PowerData {
  PowerName: string       // full power name, e.g. "Blaster_Ranged.Fire_Blast.Fire_Blast"
  Level: int               // level taken
  StatInclude: bool
  ProcInclude: bool
  VariableValue: int
  InherentSlotsUsed: int
  SubPowerEntries: SubPowerData[]
  SlotEntries: SlotData[]
}

SlotData {
  Level: int
  IsInherent: bool
  Enhancement: EnhancementData?       // primary slotted enhancement
  FlippedEnhancement: EnhancementData?  // "flipped" alt slot, if any
}

EnhancementData {
  Uid: string             // stable enhancement identifier — look this up in the enhancement DB
  Grade: string            // e.g. "None" for IOs; relevant for training/DO/SO grades
  IoLevel: int
  RelativeLevel: string    // e.g. "Even"
  Obtained: bool
}
```

Encoding (for later, if we ever want to *export* a build from our own app back into Mids-compatible form) is just the reverse: `JsonConvert.SerializeObject` → UTF8 bytes → Brotli-compress → base64 → wrap into the `|MBD;...|` header/body format.

**Legacy format:** an older `MxDz` format also exists (`^\|MxDz;[0-9]+;[0-9]+;[0-9]+;HEX;\|`), built with a custom `ModernZlib.CompressChunk` + hex encoding instead of Brotli + base64 (see `MidsCharacterFileFormat.cs`). Lower priority — only needed if we want to support very old forum-shared builds.

## 2. Power/enhancement database

Source: `MidsReborn/Databases/Rebirth/*.mhd`.

These are **not** encrypted or obfuscated. Inspecting the raw bytes:

- `SetTypes.mhd` — plain tab-delimited text, trivially parseable.
- `EnhDB.mhd` (~440KB) — a header string (`" Mids Reborn Enhancement Database"`) followed by length-prefixed strings and packed fields — the shape produced by .NET's `BinaryWriter` (`Write(string)` writes a 7-bit-encoded length prefix then UTF-8 bytes; numeric fields are written as raw little-endian primitives in a fixed order).
- `I12.mhd` (~26MB, the main powers database) — same style: header string, version string (`"2023.7.445"` in this checkout), a timestamp, and then a record structure with markers like `"BEGIN:ARCHETYPES"` delimiting sections.

Because the reader/writer code is open source, the **exact field order is knowable** — it isn't guesswork, it's whatever `DatabaseAPI.cs` (and related loader classes) read in sequence. Two viable approaches, in order of preference:

1. Port the field-read order from `DatabaseAPI.cs` into a small Python (or Node) parser that reads the `.mhd` files directly and emits clean JSON.
2. Write a minimal .NET console app that references Mids Reborn's own `DatabaseAPI`/model classes to load the database, then serializes it to JSON — lets their code do the parsing, we just dump the result. Simpler to get exactly right, but ties a build step to .NET tooling.

Either way, output should be a versioned JSON dump (powers, powersets, archetypes, enhancements, enhancement sets + their bonus tables, ED schedule tables, global mods) that gets regenerated whenever Homecoming ships a patch and Mids ships an updated database.

**Licensing:** Mids Reborn is GPL-3.0. A converter that ports or reuses its parsing logic should stay GPL-compatible if ever distributed publicly.

## 3. City of Data (cod.uberguy.net) — reference only, not a pipeline source

City of Data is entirely client-side, backed by static per-power JSON files (e.g. `https://cod.uberguy.net/homecoming/powers/index.json`), but:
- Its own maintainer describes it as built for browsing, not bulk access ("every power is sent as a distinct data file").
- Its `robots.txt` disallows automated fetching.

Useful as a human-eyeball cross-check when validating numbers we've parsed from Mids' database, not as an automated source.

## Net assessment

Both of the hard risks identified during brainstorming — "can we read a shared build?" and "can we get real, current game numbers without hand-maintaining them?" — check out as tractable using Mids Reborn's own open-source project as the foundation for both.