Better readme (#173)

* Adds better docs to README.md
* Adds tar, gzip to bundle.py checked dependencies.
* Adds fancy rendering of .data/chip-families.svg in output for embedding.

.gitignore

@@ -6,3 +6,4 @@ microzig-deploy/
 .lldbinit
 .direnv/
 __pycache__/
+.venv

README.md

@@ -1,6 +1,6 @@
 # ![MicroZig Logo](design/logo-text-auto.svg)
 
-[![Chat](https://img.shields.io/discord/824493524413710336.svg?logo=discord)](link=https://discord.gg/ShUWykk38X)
+[![Chat](https://img.shields.io/discord/824493524413710336.svg?logo=discord)](https://discord.gg/ShUWykk38X)
 [![Downloads](https://img.shields.io/badge/Zig_Package-Download-blue)](https://downloads.microzig.tech/)
 [![Continuous Integration](https://github.com/ZigEmbeddedGroup/microzig/actions/workflows/build.yml/badge.svg)](https://github.com/ZigEmbeddedGroup/microzig/actions/workflows/build.yml)
 
@@ -10,10 +10,28 @@
 
 0.11.0
 
-## Contributing
+## Getting Started With MicroZig
+
+### I Want To Use MicroZig
+
+**IMPORTANT:** You don't have to clone this repository to get started!
+
+MicroZig uses a monorepo architecture, but provides a lot of different packages. If you just want to get started, head over to [downloads.microzig.tech](https://downloads.microzig.tech/) and download an example for the chip family you desire.
+
+We support several chip families like the [RP2 family by RaspberryPi Foundation](https://www.raspberrypi.com/products/rp2040/), [STM32 by STMicroelectronics](https://www.st.com/content/st_com/en.html), and many others.
+
+Unpack the example, and run `zig build` in the resulting example folder gives you `zig-out/firmware` which contains the resulting files.
+
+Right now, you gotta figure out how to flash the MCU yourself, but as people say: Google is your friend. But you can also ask for help [on our Discord server](https://discord.gg/ShUWykk38X).
+
+### I Want To Contribute To MicroZig
+
+**IMPORTANT:** Developer experience is degraded right now, and not really good. Windows isn't really a supported dev target and you got to expect some friction. [There's a project for improving DX, feel free to grab tasks from there!](https://github.com/orgs/ZigEmbeddedGroup/projects/4)
 
 Please see the [project page](https://github.com/orgs/ZigEmbeddedGroup/projects/1/views/1), it’s used as a place to brainstorm and organize work in ZEG. There will be issues marked as good first issue or drafts for larger ideas that need scoping/breaking ground on.
 
+More words on contribution and development on MicroZig are [further down below](#developing).
+
 ## Introduction
 
 This repo contains the infrastructure for getting started in an embedded Zig project; it "gets you to main()". Specifically, it offers:
@@ -25,12 +43,6 @@ This repo contains the infrastructure for getting started in an embedded Zig pro
 * device drivers for interacting with external hardware
 * an uncomplicated method to define xref:interrupts[interrupts]
 
-## Getting Started
-
-Search for your chip family in [the examples](https://downloads.microzig.tech/examples/) and get the archive.
-
-You can easily get started based on that.
-
 ## Design
 
 For MicroZig internals please see the [Design Document](docs/design.adoc).
@@ -41,10 +53,19 @@ Right now, the developer experience is not optimal due to 0.11 not really suppor
 
 If you want to test your changes, you gotta to the following:
 
-**Step 1:** Install required python pacakges:
+**Step 1:** Install required python packages, either systemwide or via a [virtual environment](https://docs.python.org/3/library/venv.html):
 
 ```sh-session
+# systemwide:
+[user@host] microzig-monorepo/ $ pip install -r tools/requirements.txt
+[user@host] microzig-monorepo/ $ 
+
+# using virtual environments:
+[user@host] microzig-monorepo/ $ python3 -m venv .venv
+[user@host] microzig-monorepo/ $ . .venv/bin/activate # on linux, macos
+[user@host] microzig-monorepo/ $ . .venv/Scripts/activate # on windows
 [user@host] microzig-monorepo/ $ pip3 install -r tools/requirements.txt
+[user@host] microzig-monorepo/ $ 
 ```
 
 **Step 2:** Create a deployment for local usage:
@@ -83,7 +104,11 @@ start fetching packages from this.
 
 Now you can use curl to fetch the packages, or you can just create a local development project.
 
-**Step 4:** You can use the tool `tools/patch-build-zon.py` to patch/upgrade your development project inplace:
+**Step 4:** Create a local test environment
+
+This is basically done by unpacking an example from the `./microzig-deploy/examples` folder, and starting to test changes.
+As the `build.zig.zon` has to be updated after running `./tools/bundle.py` again, there's a script that helps here:
+`tools/patch-build-zon.py` can be used to patch/upgrade your development project inplace based on what it finds in `./microzig-deploy`:
 
 ```sh-session
 [user@host] microzig-monorepo/ $ python3 ./tools/patch-build-zon.py /tmp/dev-project/build.zig.zon
@@ -93,6 +118,8 @@ Updating hash of http://localhost:8080/packages/board-support/stmicro/stm32.tar.
 [user@host] microzig-monorepo/ $
 ```
 
+Both compiling the local example and updating the `build.zig.zon` requires running the local development server.
+
 ## Repository structure
 
 * `build/` contains the build components of MicroZig.

board-support/espressif-esp/microzig-package.json

@@ -1,4 +1,4 @@
 {
-  "package_name": "espressif/esp",
-  "package_type": "board-support"
-}
+    "package_name": "espressif/esp",
+    "package_type": "board-support"
+}

tools/bundle.py

@@ -7,7 +7,7 @@
 #
 
 
-import sys, os, datetime, re, shutil, json, hashlib
+import io, sys, os, datetime, re, shutil, json, hashlib, html 
 from pathlib import Path, PurePosixPath
 from dataclasses import dataclass, field 
 from dataclasses_json import dataclass_json, config as  dcj_config, Exclude as JsonExclude
@@ -37,6 +37,8 @@ ALL_FILES_DIR=".data"
 REQUIRED_TOOLS = [
     "zig",
     "git",
+    "tar",
+    "gzip",
 ]
 REQUIRED_ZIG_VERSION="0.11.0"
 
@@ -94,7 +96,7 @@ class PackageConfiguration:
 
     download_url: Optional[str] = field(default=None)
 
-    microzig: Optional[Any] = field(default=None)
+    microzig: Optional[Any] = field(default=None) # optional configuration field with microzig-specific options
 
     # inner fields:
     # package_dir: Path = field(default=None, metadata = dcj_config(exclude=JsonExclude.ALWAYS))
@@ -259,7 +261,7 @@ def main():
 
     print("preparing environment...")
 
-    deploy_target = create_output_directory(REPO_ROOT)    
+    deploy_target = create_output_directory(REPO_ROOT)
 
     # Some generic meta information:
     batch_timestamp = get_batch_timestamp()
@@ -276,6 +278,9 @@ def main():
     cache_dir = cache_root / "microzig"
     cache_dir.mkdir(exist_ok=True)
 
+    detail_files_dir = deploy_target / ALL_FILES_DIR
+    detail_files_dir.mkdir(exist_ok=True)
+
     # Prepare `.gitignore` pattern matcher:
     global_ignore_spec = pathspec.PathSpec.from_lines(
         pathspec.patterns.GitWildMatchPattern, 
@@ -528,9 +533,215 @@ def main():
     with (deploy_target / "index.json").open("w") as f:
         f.write(PackageIndexSchema.dumps(index))
 
-    # TODO: Verify that each package can be unpacked and built
 
+    with (detail_files_dir / "chip-families.svg").open("w") as f:
+        ICON_YES = TableIcon(
+            glyph="✅",
+            path="M10,17L5,12L6.41,10.58L10,14.17L17.59,6.58L19,8M19,3H5C3.89,3 3,3.89 3,5V19A2,2 0 0,0 5,21H19A2,2 0 0,0 21,19V5C21,3.89 20.1,3 19,3Z",
+            color="#0f0",
+        )
+        ICON_NO = TableIcon(
+            glyph="❌",
+            path="M19,6.41L17.59,5L12,10.59L6.41,5L5,6.41L10.59,12L5,17.59L6.41,19L12,13.41L17.59,19L19,17.59L13.41,12L19,6.41Z",
+            color="#f44336",
+        )
+        ICON_WIP = TableIcon(
+            glyph="🛠",
+            path="M13.78 15.3L19.78 21.3L21.89 19.14L15.89 13.14L13.78 15.3M17.5 10.1C17.11 10.1 16.69 10.05 16.36 9.91L4.97 21.25L2.86 19.14L10.27 11.74L8.5 9.96L7.78 10.66L6.33 9.25V12.11L5.63 12.81L2.11 9.25L2.81 8.55H5.62L4.22 7.14L7.78 3.58C8.95 2.41 10.83 2.41 12 3.58L9.89 5.74L11.3 7.14L10.59 7.85L12.38 9.63L14.2 7.75C14.06 7.42 14 7 14 6.63C14 4.66 15.56 3.11 17.5 3.11C18.09 3.11 18.61 3.25 19.08 3.53L16.41 6.2L17.91 7.7L20.58 5.03C20.86 5.5 21 6 21 6.63C21 8.55 19.45 10.1 17.5 10.1Z",
+            color="#82aec0",
+        )
+        ICON_EXPERIMENTAL = TableIcon(
+            glyph="🧪",
+            path="M7,2V4H8V18A4,4 0 0,0 12,22A4,4 0 0,0 16,18V4H17V2H7M11,16C10.4,16 10,15.6 10,15C10,14.4 10.4,14 11,14C11.6,14 12,14.4 12,15C12,15.6 11.6,16 11,16M13,12C12.4,12 12,11.6 12,11C12,10.4 12.4,10 13,10C13.6,10 14,10.4 14,11C14,11.6 13.6,12 13,12M14,7H10V4H14V7Z",
+            color="#2ac9b4"
+        )
+        ICON_UNKNOWN = TableIcon(
+            glyph="❓",
+            path="M10,19H13V22H10V19M12,2C17.35,2.22 19.68,7.62 16.5,11.67C15.67,12.67 14.33,13.33 13.67,14.17C13,15 13,16 13,17H10C10,15.33 10,13.92 10.67,12.92C11.33,11.92 12.67,11.33 13.5,10.67C15.92,8.43 15.32,5.26 12,5A3,3 0 0,0 9,8H6A6,6 0 0,1 12,2Z",
+            color="#ed4034"
+        )
+        
+        def bsp_info(bsp: PackageConfiguration):
+
+            targets = bsp.microzig["board-support"]["targets"]
+
+            num_chips = len([
+                1
+                for tgt
+                in targets
+                if tgt["board"] is None
+            ])
+            num_boards = len(targets) - num_chips
+
+            any_hal = any(
+                tgt["features"]["hal"]
+                for tgt
+                in targets
+            )
+
+            all_hal = all(
+                tgt["features"]["hal"]
+                for tgt
+                in targets
+            )
 
+            hal_support = ICON_NO
+            if all_hal:
+                hal_support = ICON_YES
+            elif any_hal:
+                hal_support = ICON_WIP
+
+            examples = packages.get(f"examples:{bsp.package_name}", None)
+
+
+            return [
+                bsp.package_name, # Chip Family
+                num_chips,  # Chips
+                num_boards,  # Boards
+                hal_support,  # HAL
+                TableLink(href=f"https://downloads.microzig.tech/examples/{bsp.package_name}.tar.gz", content="Download") if examples is not None else "", # Examples
+            ]
+
+        render_table(
+            target=f,
+            columns=[
+                TableColumn(title="Chip Family", width=200),
+                TableColumn(title="Chips", width=70),
+                TableColumn(title="Boards", width=70),
+                TableColumn(title="HAL", width=50),
+                TableColumn(title="Examples", width=100),
+                # TableColumn(title="Abstractions", width=120),
+            ],
+            rows = [
+                bsp_info(bsp)
+                for key, bsp
+                in sorted(packages.items())
+                if bsp.package_type == PackageType.board_support
+            ],
+        ) 
+
+@dataclass
+class TableLink:
+    href: str
+    content: any
+
+@dataclass
+class TableIcon:
+    glyph: str
+    path: str
+    color: str = field(default="#fff")
+
+@dataclass
+class TableColumn:
+    title: str
+    width: int
+
+def render_table(target: io.IOBase, columns: list[TableColumn], rows: list[list[str]]):
+
+    cell_height = 25
+
+    total_width = sum(col.width for col in columns)
+    total_height = cell_height * (1 + len(rows))
+
+    target.write('<?xml version="1.0" encoding="UTF-8" standalone="no"?>')
+    target.write('<svg')
+    target.write(' width="%s"' % (total_width))
+    target.write(' height="%s"' % (total_height))
+    target.write(' viewBox="0 0 %s %s"' % (total_width, total_height))
+    target.write(' version="1.1"')
+    target.write(' xmlns="http://www.w3.org/2000/svg">')
+    target.write("""<style>
+        svg {
+            background-color: #0d1117;
+        }
+        rect {
+            fill: none;
+            stroke: #30363d;
+            stroke-width: 1;
+            stroke-linecap: round;
+            stroke-linejoin: round;
+            stroke-opacity: 1;
+        }
+        rect.odd {
+            fill: #161b22;
+        }
+        text {
+            font-style: normal;
+            font-variant: normal;
+            font-weight: normal;
+            font-stretch: normal;
+            font-size: 20;
+            line-height:1.25;
+            font-family: sans-serif;
+            font-variant-ligatures: normal;
+            font-variant-caps: normal;
+            font-variant-numeric: normal;
+            stroke-width: 1;
+            fill: #e6edf3;
+        }
+        text.heading {
+            font-weight: bold;
+        }
+        a {
+            fill: #2c81f7;
+        }
+   </style>""")
+    target.write('<g>')
+
+    x = 0
+    y = 0
+
+    def valueout(content: any):
+
+        if isinstance(content, TableLink):
+            target.write(f'<a target="_blank" href="{content.href}">')
+            valueout(content.content)
+            target.write('</a>')
+        elif isinstance(content, TableIcon):
+            target.write(content.glyph)
+        else:
+            target.write(html.escape(str(content)))
+
+    def cellout(col: TableColumn, content: any, is_heading: bool = False):
+        nonlocal x, y 
+        
+        target.write('<rect')
+        target.write(f' width="{col.width}"')
+        target.write(f' height="{cell_height}"')
+        target.write(f' x="{x}"')
+        target.write(f' y="{y}"')
+        target.write('/>')
+
+        target.write('<text')
+        if is_heading:
+            target.write(' class="heading"')
+        target.write(f' space="preserve"')
+        target.write(f' x="{x+5}"')
+        target.write(f' y="{y+cell_height-5}"')
+        target.write('>')
+        valueout(content)
+        target.write("</text>")
+
+        x += col.width
+
+    def endrow():
+        nonlocal x, y
+        x = 0
+        y += cell_height
+
+    for col in columns:
+        cellout(col, col.title, is_heading=True)
+    endrow()
+
+    for row in rows:
+
+        for col, cell in zip(columns, row):
+            cellout(col, cell)
+        endrow()
+
+    target.write("""  </g>
+</svg>
+""")
 
 if __name__ == "__main__":
     main()

tools/requirements.txt

@@ -1,5 +1,5 @@
 dataclasses_json==0.6.3
-marshmallow
-typing-inspect
-semver
+marshmallow>=3.20.0
+typing-inspect>=0.9.0
+semver>=3.0.0
 pathspec>=0.12.0