{
  "type": "excalidraw",
  "version": 2,
  "source": "https://excalidraw.com",
  "elements": [
    {
      "id": "bg-left",
      "type": "rectangle",
      "x": 30,
      "y": 90,
      "width": 500,
      "height": 860,
      "angle": 0,
      "strokeColor": "#93c5fd",
      "backgroundColor": "#dbeafe",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 28,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1000,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "bg-middle",
      "type": "rectangle",
      "x": 560,
      "y": 90,
      "width": 650,
      "height": 860,
      "angle": 0,
      "strokeColor": "#a78bfa",
      "backgroundColor": "#ede9fe",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 25,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1001,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "bg-right",
      "type": "rectangle",
      "x": 1240,
      "y": 90,
      "width": 500,
      "height": 860,
      "angle": 0,
      "strokeColor": "#34d399",
      "backgroundColor": "#dcfce7",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 25,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1002,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "bg-bottom",
      "type": "rectangle",
      "x": 30,
      "y": 980,
      "width": 1710,
      "height": 280,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fffbeb",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 35,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1003,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "title",
      "type": "text",
      "x": 385,
      "y": 24,
      "width": 688.2,
      "height": 37.5,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1004,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "FlashAttention 面试讲解画布：从 v1 到 v4 的优化主线",
      "fontSize": 30,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "FlashAttention 面试讲解画布：从 v1 到 v4 的优化主线",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "subtitle",
      "type": "text",
      "x": 455,
      "y": 62,
      "width": 714.24,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1005,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "核心问题：标准 Attention 不缺 FLOPs，先卡在显存 IO，后卡在如何持续喂满新 GPU 的 Tensor Core",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "核心问题：标准 Attention 不缺 FLOPs，先卡在显存 IO，后卡在如何持续喂满新 GPU 的 Tensor Core",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "left-title",
      "type": "text",
      "x": 65,
      "y": 112,
      "width": 177.32,
      "height": 27.5,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1006,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "一、先讲清楚它到底解决什么",
      "fontSize": 22,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "一、先讲清楚它到底解决什么",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "std-box",
      "type": "rectangle",
      "x": 65,
      "y": 155,
      "width": 420,
      "height": 130,
      "angle": 0,
      "strokeColor": "#ef4444",
      "backgroundColor": "#fee2e2",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1007,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "std-text",
      "type": "text",
      "x": 90,
      "y": 178,
      "width": 274.04,
      "height": 85,
      "angle": 0,
      "strokeColor": "#7f1d1d",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1008,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "标准 Attention\nS = QK^T 生成 N x N 分数矩阵\nP = softmax(S) 再乘 V\n问题：S 和 P 写回 HBM，长序列显存和带宽都爆",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "标准 Attention\nS = QK^T 生成 N x N 分数矩阵\nP = softmax(S) 再乘 V\n问题：S 和 P 写回 HBM，长序列显存和带宽都爆",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "flash-box",
      "type": "rectangle",
      "x": 65,
      "y": 335,
      "width": 420,
      "height": 155,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "#bfdbfe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1009,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "flash-text",
      "type": "text",
      "x": 90,
      "y": 360,
      "width": 316.2,
      "height": 106.25,
      "angle": 0,
      "strokeColor": "#1e3a8a",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1010,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "FlashAttention 的一句话答案\n不近似，不改变数学结果\n把 K/V 分块搬进 SRAM\n边算 QK，边做 online softmax，边累加 PV\n避免物化完整 N x N 注意力矩阵",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "FlashAttention 的一句话答案\n不近似，不改变数学结果\n把 K/V 分块搬进 SRAM\n边算 QK，边做 online softmax，边累加 PV\n避免物化完整 N x N 注意力矩阵",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "std-to-flash",
      "type": "arrow",
      "x": 270,
      "y": 292,
      "width": 0,
      "height": 35,
      "angle": 0,
      "strokeColor": "#3b82f6",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 3,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1011,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          0,
          35
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "formula-box",
      "type": "rectangle",
      "x": 65,
      "y": 540,
      "width": 420,
      "height": 155,
      "angle": 0,
      "strokeColor": "#10b981",
      "backgroundColor": "#d1fae5",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1012,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "formula-text",
      "type": "text",
      "x": 90,
      "y": 565,
      "width": 231.88,
      "height": 106.25,
      "angle": 0,
      "strokeColor": "#064e3b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1013,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "online softmax 关键状态\nm：当前块最大值，用于数值稳定\nl：exp 和的缩放累加\nO：输出累加器\n新块到来时重缩放旧 O 和 l，再加新块贡献",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "online softmax 关键状态\nm：当前块最大值，用于数值稳定\nl：exp 和的缩放累加\nO：输出累加器\n新块到来时重缩放旧 O 和 l，再加新块贡献",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "what-not-box",
      "type": "rectangle",
      "x": 65,
      "y": 735,
      "width": 420,
      "height": 165,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fef3c7",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1014,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "what-not-text",
      "type": "text",
      "x": 90,
      "y": 760,
      "width": 379.44,
      "height": 106.25,
      "angle": 0,
      "strokeColor": "#78350f",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1015,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "面试先避坑\n不是 sparse attention，也不是 low-rank 近似\n时间复杂度仍是 O(N^2 d)\n显存占用从存 N^2 中间矩阵转向 O(Nd)\n优势来自 IO-aware kernel fusion 和 tiling",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "面试先避坑\n不是 sparse attention，也不是 low-rank 近似\n时间复杂度仍是 O(N^2 d)\n显存占用从存 N^2 中间矩阵转向 O(Nd)\n优势来自 IO-aware kernel fusion 和 tiling",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "middle-title",
      "type": "text",
      "x": 595,
      "y": 112,
      "width": 204.6,
      "height": 27.5,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1016,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "二、v1 到 v4 的优化演进",
      "fontSize": 22,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "二、v1 到 v4 的优化演进",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "timeline-line",
      "type": "line",
      "x": 615,
      "y": 210,
      "width": 540,
      "height": 0,
      "angle": 0,
      "strokeColor": "#6b7280",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 3,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1017,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          540,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null
    },
    {
      "id": "v1-dot",
      "type": "ellipse",
      "x": 625,
      "y": 190,
      "width": 36,
      "height": 36,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "#bfdbfe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1018,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v1-label",
      "type": "text",
      "x": 577,
      "y": 240,
      "width": 168.64,
      "height": 60,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1019,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "2022\nIO-aware tiling\nHBM <-> SRAM 少搬数据",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "center",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "2022\nIO-aware tiling\nHBM <-> SRAM 少搬数据",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v2-dot",
      "type": "ellipse",
      "x": 795,
      "y": 190,
      "width": 36,
      "height": 36,
      "angle": 0,
      "strokeColor": "#16a34a",
      "backgroundColor": "#bbf7d0",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1020,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v2-label",
      "type": "text",
      "x": 747,
      "y": 240,
      "width": 178.56,
      "height": 60,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1021,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "2023\n更好并行划分\n减少非 matmul 和共享内存通信",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "center",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "2023\n更好并行划分\n减少非 matmul 和共享内存通信",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v3-dot",
      "type": "ellipse",
      "x": 965,
      "y": 190,
      "width": 36,
      "height": 36,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "#e9d5ff",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1022,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v3-label",
      "type": "text",
      "x": 917,
      "y": 240,
      "width": 158.72,
      "height": 80,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1023,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "2024\nHopper 专用\nTMA + WGMMA 异步流水\nFP8",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "center",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "2024\nHopper 专用\nTMA + WGMMA 异步流水\nFP8",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v4-dot",
      "type": "ellipse",
      "x": 1135,
      "y": 190,
      "width": 36,
      "height": 36,
      "angle": 0,
      "strokeColor": "#ea580c",
      "backgroundColor": "#fed7aa",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1024,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v4-label",
      "type": "text",
      "x": 1087,
      "y": 240,
      "width": 138.88,
      "height": 80,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1025,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "2026\nBlackwell 专用强化\n全异步 MMA + TMEM\n更激进 softmax",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "center",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "2026\nBlackwell 专用强化\n全异步 MMA + TMEM\n更激进 softmax",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v1-card",
      "type": "rectangle",
      "x": 595,
      "y": 355,
      "width": 280,
      "height": 175,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "#bfdbfe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1026,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v1-card-head",
      "type": "text",
      "x": 613,
      "y": 371,
      "width": 301.32,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1027,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v1：IO-aware exact attention",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v1：IO-aware exact attention",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v1-card-body",
      "type": "text",
      "x": 613,
      "y": 403,
      "width": 232.5,
      "height": 75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1028,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "瓶颈：HBM 读写远慢于 SRAM\n核心：Q 分块，K/V 分块，块内计算 S、P、O\n不保存 S/P，反向只保存少量统计量再重算\n面试句：用更多计算重算换少写显存，精确结果不变",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "瓶颈：HBM 读写远慢于 SRAM\n核心：Q 分块，K/V 分块，块内计算 S、P、O\n不保存 S/P，反向只保存少量统计量再重算\n面试句：用更多计算重算换少写显存，精确结果不变",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v2-card",
      "type": "rectangle",
      "x": 905,
      "y": 355,
      "width": 280,
      "height": 175,
      "angle": 0,
      "strokeColor": "#16a34a",
      "backgroundColor": "#bbf7d0",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1029,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v2-card-head",
      "type": "text",
      "x": 923,
      "y": 371,
      "width": 100.44,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#16a34a",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1030,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v2：更好的并行度",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v2：更好的并行度",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v2-card-body",
      "type": "text",
      "x": 923,
      "y": 403,
      "width": 353.4,
      "height": 75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1031,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "瓶颈：v1 还没接近 GEMM，occupancy 和 warp 通信不理想\n核心：跨 thread block 并行单个 head\nwarp 内重新分工，减少 shared memory 往返\n面试句：算法思想不变，主要优化 work partitioning",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "瓶颈：v1 还没接近 GEMM，occupancy 和 warp 通信不理想\n核心：跨 thread block 并行单个 head\nwarp 内重新分工，减少 shared memory 往返\n面试句：算法思想不变，主要优化 work partitioning",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v3-card",
      "type": "rectangle",
      "x": 595,
      "y": 590,
      "width": 280,
      "height": 175,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "#e9d5ff",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1032,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v3-card-head",
      "type": "text",
      "x": 613,
      "y": 606,
      "width": 156.24,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1033,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v3：Hopper 异步流水",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v3：Hopper 异步流水",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v3-card-body",
      "type": "text",
      "x": 613,
      "y": 638,
      "width": 409.2,
      "height": 75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1034,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "瓶颈：H100 上 Tensor Core 很快，load 和 softmax 会拖后腿\n核心：TMA 搬运 + WGMMA 计算异步化\nproducer-consumer warp specialization\nmatmul 与 softmax 交错，FP8 低精度并控误差",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "瓶颈：H100 上 Tensor Core 很快，load 和 softmax 会拖后腿\n核心：TMA 搬运 + WGMMA 计算异步化\nproducer-consumer warp specialization\nmatmul 与 softmax 交错，FP8 低精度并控误差",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v4-card",
      "type": "rectangle",
      "x": 905,
      "y": 590,
      "width": 280,
      "height": 175,
      "angle": 0,
      "strokeColor": "#ea580c",
      "backgroundColor": "#fed7aa",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1035,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "v4-card-head",
      "type": "text",
      "x": 923,
      "y": 606,
      "width": 200.88,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#ea580c",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1036,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v4：Blackwell 非对称扩展",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v4：Blackwell 非对称扩展",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v4-card-body",
      "type": "text",
      "x": 923,
      "y": 638,
      "width": 483.6,
      "height": 75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1037,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "瓶颈：B200 Tensor Core 翻倍，但 shared memory 和 exp 单元没同步翻倍\n核心：更深 fully async MMA pipeline，大 tile\nTMEM 存中间结果，2-CTA MMA 优化反向\n软件 exp 与条件 softmax rescale 降低非 matmul 开销",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "瓶颈：B200 Tensor Core 翻倍，但 shared memory 和 exp 单元没同步翻倍\n核心：更深 fully async MMA pipeline，大 tile\nTMEM 存中间结果，2-CTA MMA 优化反向\n软件 exp 与条件 softmax rescale 降低非 matmul 开销",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "v1-v2-arrow",
      "type": "arrow",
      "x": 870,
      "y": 442,
      "width": 35,
      "height": 0,
      "angle": 0,
      "strokeColor": "#3b82f6",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1038,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          35,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "v2-v3-arrow",
      "type": "arrow",
      "x": 1030,
      "y": 532,
      "width": -260,
      "height": 55,
      "angle": 0,
      "strokeColor": "#3b82f6",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1039,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          -260,
          55
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "v3-v4-arrow",
      "type": "arrow",
      "x": 870,
      "y": 675,
      "width": 35,
      "height": 0,
      "angle": 0,
      "strokeColor": "#3b82f6",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1040,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          35,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "lens-box",
      "type": "rectangle",
      "x": 595,
      "y": 810,
      "width": 590,
      "height": 105,
      "angle": 0,
      "strokeColor": "#64748b",
      "backgroundColor": "#f8fafc",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "dashed",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1041,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "lens-text",
      "type": "text",
      "x": 620,
      "y": 832,
      "width": 825.84,
      "height": 45,
      "angle": 0,
      "strokeColor": "#334155",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1042,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "版本主线记忆法\nv1：少搬数据  ->  v2：多开并行  ->  v3：异步重叠  ->  v4：面向 Blackwell 重排 pipeline，削掉非矩阵瓶颈",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "版本主线记忆法\nv1：少搬数据  ->  v2：多开并行  ->  v3：异步重叠  ->  v4：面向 Blackwell 重排 pipeline，削掉非矩阵瓶颈",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "right-title",
      "type": "text",
      "x": 1275,
      "y": 112,
      "width": 122.76,
      "height": 27.5,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1043,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "三、面试常考点速答",
      "fontSize": 22,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "三、面试常考点速答",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa1-box",
      "type": "rectangle",
      "x": 1275,
      "y": 155,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#dc2626",
      "backgroundColor": "#fee2e2",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1044,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa1-q",
      "type": "text",
      "x": 1293,
      "y": 167,
      "width": 168.64,
      "height": 20,
      "angle": 0,
      "strokeColor": "#dc2626",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1045,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q1 为什么显存降了但复杂度没变？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q1 为什么显存降了但复杂度没变？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa1-a",
      "type": "text",
      "x": 1293,
      "y": 194,
      "width": 390.6,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1046,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "仍要算所有 QK^T 点积，FLOPs 还是二次；省的是不把 S/P 写回 HBM。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "仍要算所有 QK^T 点积，FLOPs 还是二次；省的是不把 S/P 写回 HBM。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa2-box",
      "type": "rectangle",
      "x": 1275,
      "y": 265,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "#dbeafe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1047,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa2-q",
      "type": "text",
      "x": 1293,
      "y": 277,
      "width": 248,
      "height": 20,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1048,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q2 online softmax 怎么保证精确？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q2 online softmax 怎么保证精确？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa2-a",
      "type": "text",
      "x": 1293,
      "y": 304,
      "width": 483.6,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1049,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "每块维护 max 和 sum，遇到更大 max 时把旧累加按 exp(old_m-new_m) 重缩放。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "每块维护 max 和 sum，遇到更大 max 时把旧累加按 exp(old_m-new_m) 重缩放。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa3-box",
      "type": "rectangle",
      "x": 1275,
      "y": 375,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#16a34a",
      "backgroundColor": "#dcfce7",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1050,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa3-q",
      "type": "text",
      "x": 1293,
      "y": 387,
      "width": 128.96,
      "height": 20,
      "angle": 0,
      "strokeColor": "#16a34a",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1051,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q3 反向为什么能省内存？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q3 反向为什么能省内存？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa3-a",
      "type": "text",
      "x": 1293,
      "y": 414,
      "width": 344.1,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1052,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "前向只存输出和每行统计量；反向重算局部分数和概率，避免保存完整注意力矩阵。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "前向只存输出和每行统计量；反向重算局部分数和概率，避免保存完整注意力矩阵。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa4-box",
      "type": "rectangle",
      "x": 1275,
      "y": 485,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#d97706",
      "backgroundColor": "#fef3c7",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1053,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa4-q",
      "type": "text",
      "x": 1293,
      "y": 497,
      "width": 198.4,
      "height": 20,
      "angle": 0,
      "strokeColor": "#d97706",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1054,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q4 v2 相比 v1 本质提升是什么？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q4 v2 相比 v1 本质提升是什么？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa4-a",
      "type": "text",
      "x": 1293,
      "y": 524,
      "width": 409.2,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1055,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "不是换数学公式，而是让 block、warp、head 的任务划分更适合 GPU 并行。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "不是换数学公式，而是让 block、warp、head 的任务划分更适合 GPU 并行。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa5-box",
      "type": "rectangle",
      "x": 1275,
      "y": 595,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "#f3e8ff",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1056,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa5-q",
      "type": "text",
      "x": 1293,
      "y": 607,
      "width": 208.32,
      "height": 20,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1057,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q5 v3 为什么和 Hopper 绑定？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q5 v3 为什么和 Hopper 绑定？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa5-a",
      "type": "text",
      "x": 1293,
      "y": 634,
      "width": 474.3,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1058,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "依赖 Hopper 的 TMA、WGMMA、FP8 等能力，把搬运、矩阵乘、softmax 尽量重叠。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "依赖 Hopper 的 TMA、WGMMA、FP8 等能力，把搬运、矩阵乘、softmax 尽量重叠。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa6-box",
      "type": "rectangle",
      "x": 1275,
      "y": 705,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#ea580c",
      "backgroundColor": "#ffedd5",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1059,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa6-q",
      "type": "text",
      "x": 1293,
      "y": 717,
      "width": 128.96,
      "height": 20,
      "angle": 0,
      "strokeColor": "#ea580c",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1060,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q6 v4 为什么又要改？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q6 v4 为什么又要改？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa6-a",
      "type": "text",
      "x": 1293,
      "y": 744,
      "width": 734.7,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1061,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Blackwell Tensor Core 更快，但 shared memory 和 exp 不是等比例更快，瓶颈转向 pipeline 和非 matmul。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Blackwell Tensor Core 更快，但 shared memory 和 exp 不是等比例更快，瓶颈转向 pipeline 和非 matmul。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa7-box",
      "type": "rectangle",
      "x": 1275,
      "y": 815,
      "width": 430,
      "height": 88,
      "angle": 0,
      "strokeColor": "#0284c7",
      "backgroundColor": "#e0f2fe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1062,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "qa7-q",
      "type": "text",
      "x": 1293,
      "y": 827,
      "width": 119.03999999999999,
      "height": 20,
      "angle": 0,
      "strokeColor": "#0284c7",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1063,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "Q7 什么时候收益最大？",
      "fontSize": 16,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "Q7 什么时候收益最大？",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "qa7-a",
      "type": "text",
      "x": 1293,
      "y": 854,
      "width": 465,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1064,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "长序列、大 batch 或显存带宽敏感时最明显；很短序列、特殊 mask 或不支持硬件时收益会变小。",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "长序列、大 batch 或显存带宽敏感时最明显；很短序列、特殊 mask 或不支持硬件时收益会变小。",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "bottom-title",
      "type": "text",
      "x": 65,
      "y": 1010,
      "width": 136.4,
      "height": 27.5,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1065,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "四、讲解视频分镜顺序",
      "fontSize": 22,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "四、讲解视频分镜顺序",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-1-circle",
      "type": "ellipse",
      "x": 80,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1066,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-1-num",
      "type": "text",
      "x": 95,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1067,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "1",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "1",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-1-box",
      "type": "rectangle",
      "x": 70,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1068,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-1-text",
      "type": "text",
      "x": 88,
      "y": 1148,
      "width": 269.7,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1069,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "先画标准 Attention：S 和 P 为什么占 N^2",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "先画标准 Attention：S 和 P 为什么占 N^2",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-1-arrow",
      "type": "arrow",
      "x": 128,
      "y": 1092,
      "width": 190,
      "height": 0,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1070,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          190,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "step-2-circle",
      "type": "ellipse",
      "x": 335,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1071,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-2-num",
      "type": "text",
      "x": 350,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1072,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "2",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "2",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-2-box",
      "type": "rectangle",
      "x": 325,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1073,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-2-text",
      "type": "text",
      "x": 343,
      "y": 1148,
      "width": 241.8,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1074,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "解释 GPU 内存层级：HBM 慢，SRAM 快但小",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "解释 GPU 内存层级：HBM 慢，SRAM 快但小",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-2-arrow",
      "type": "arrow",
      "x": 383,
      "y": 1092,
      "width": 190,
      "height": 0,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1075,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          190,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "step-3-circle",
      "type": "ellipse",
      "x": 590,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1076,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-3-num",
      "type": "text",
      "x": 605,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1077,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "3",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "3",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-3-box",
      "type": "rectangle",
      "x": 580,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1078,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-3-text",
      "type": "text",
      "x": 598,
      "y": 1148,
      "width": 325.5,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1079,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v1：tiling + online softmax，精确但不物化矩阵",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v1：tiling + online softmax，精确但不物化矩阵",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-3-arrow",
      "type": "arrow",
      "x": 638,
      "y": 1092,
      "width": 190,
      "height": 0,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1080,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          190,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "step-4-circle",
      "type": "ellipse",
      "x": 845,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1081,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-4-num",
      "type": "text",
      "x": 860,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1082,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "4",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "4",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-4-box",
      "type": "rectangle",
      "x": 835,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1083,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-4-text",
      "type": "text",
      "x": 853,
      "y": 1148,
      "width": 362.7,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1084,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v2：work partitioning，让 GPU occupancy 上去",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v2：work partitioning，让 GPU occupancy 上去",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-4-arrow",
      "type": "arrow",
      "x": 893,
      "y": 1092,
      "width": 190,
      "height": 0,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1085,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          190,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "step-5-circle",
      "type": "ellipse",
      "x": 1100,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1086,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-5-num",
      "type": "text",
      "x": 1115,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1087,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "5",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "5",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-5-box",
      "type": "rectangle",
      "x": 1090,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1088,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-5-text",
      "type": "text",
      "x": 1108,
      "y": 1148,
      "width": 204.6,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1089,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v3：Hopper 异步流水，搬运和计算重叠",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v3：Hopper 异步流水，搬运和计算重叠",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-5-arrow",
      "type": "arrow",
      "x": 1148,
      "y": 1092,
      "width": 190,
      "height": 0,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1090,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "points": [
        [
          0,
          0
        ],
        [
          190,
          0
        ]
      ],
      "startBinding": null,
      "endBinding": null,
      "startArrowhead": null,
      "endArrowhead": "arrow"
    },
    {
      "id": "step-6-circle",
      "type": "ellipse",
      "x": 1355,
      "y": 1070,
      "width": 44,
      "height": 44,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fde68a",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1091,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-6-num",
      "type": "text",
      "x": 1370,
      "y": 1080,
      "width": 80,
      "height": 25,
      "angle": 0,
      "strokeColor": "#92400e",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1092,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "6",
      "fontSize": 20,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "6",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "step-6-box",
      "type": "rectangle",
      "x": 1345,
      "y": 1130,
      "width": 210,
      "height": 75,
      "angle": 0,
      "strokeColor": "#f59e0b",
      "backgroundColor": "#fff7ed",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1093,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "step-6-text",
      "type": "text",
      "x": 1363,
      "y": 1148,
      "width": 334.8,
      "height": 18.75,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1094,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "v4：Blackwell 非对称瓶颈，TMEM 与 softmax 重排",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "v4：Blackwell 非对称瓶颈，TMEM 与 softmax 重排",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "source-box",
      "type": "rectangle",
      "x": 1245,
      "y": 1010,
      "width": 465,
      "height": 210,
      "angle": 0,
      "strokeColor": "#94a3b8",
      "backgroundColor": "#f8fafc",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "dashed",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "roundness": {
        "type": 3
      },
      "seed": 1095,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false
    },
    {
      "id": "source-title",
      "type": "text",
      "x": 1270,
      "y": 1030,
      "width": 122.76,
      "height": 22.5,
      "angle": 0,
      "strokeColor": "#475569",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1096,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "可在视频口播的来源提示",
      "fontSize": 18,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "可在视频口播的来源提示",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "source-text",
      "type": "text",
      "x": 1270,
      "y": 1065,
      "width": 474.3,
      "height": 112.5,
      "angle": 0,
      "strokeColor": "#374151",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 1097,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "主要依据：\narXiv 2205.14135 FlashAttention v1\narXiv 2307.08691 FlashAttention-2\narXiv 2407.08608 FlashAttention-3\narXiv 2603.05451 FlashAttention-4\nNVIDIA 2024 技术博客与 PyTorch 2026 FlexAttention/FA4 博客",
      "fontSize": 15,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "主要依据：\narXiv 2205.14135 FlashAttention v1\narXiv 2307.08691 FlashAttention-2\narXiv 2407.08608 FlashAttention-3\narXiv 2603.05451 FlashAttention-4\nNVIDIA 2024 技术博客与 PyTorch 2026 FlexAttention/FA4 博客",
      "autoResize": true,
      "lineHeight": 1.25
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-bg",
      "type": "rectangle",
      "x": 80,
      "y": 1340.0,
      "width": 1580,
      "height": 300,
      "angle": 0,
      "strokeColor": "#64748b",
      "backgroundColor": "#e2e8f0",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 70,
      "groupIds": [],
      "seed": 93001,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "roundness": {
        "type": 3
      }
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-title",
      "type": "text",
      "x": 108,
      "y": 1362.0,
      "width": 1524,
      "height": 38,
      "angle": 0,
      "strokeColor": "#1e40af",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 91001,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "练习题与面试题补充：FlashAttention IO-aware attention",
      "fontSize": 24,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "练习题与面试题补充：FlashAttention IO-aware attention",
      "autoResize": false,
      "lineHeight": 1.25
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-practice-box",
      "type": "rectangle",
      "x": 108,
      "y": 1418.0,
      "width": 748.0,
      "height": 190,
      "angle": 0,
      "strokeColor": "#2563eb",
      "backgroundColor": "#bfdbfe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 93002,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "roundness": {
        "type": 3
      }
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-practice-text",
      "type": "text",
      "x": 130,
      "y": 1438.0,
      "width": 725.0,
      "height": 150,
      "angle": 0,
      "strokeColor": "#243042",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 91001,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "练习题\n1. 手推一个 Q block 扫两个 K/V tile 的 online softmax 更新\n2. 画出 S tile、P tile、O tile 的生命周期\n3. 比较 v1/v2/v3/v4 在 work partition 和 pipeline 上的变化",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "练习题\n1. 手推一个 Q block 扫两个 K/V tile 的 online softmax 更新\n2. 画出 S tile、P tile、O tile 的生命周期\n3. 比较 v1/v2/v3/v4 在 work partition 和 pipeline 上的变化",
      "autoResize": false,
      "lineHeight": 1.25
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-interview-box",
      "type": "rectangle",
      "x": 884.0,
      "y": 1418.0,
      "width": 748.0,
      "height": 190,
      "angle": 0,
      "strokeColor": "#7c3aed",
      "backgroundColor": "#ddd6fe",
      "fillStyle": "solid",
      "strokeWidth": 2,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 93003,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "roundness": {
        "type": 3
      }
    },
    {
      "id": "practice-interview-panel-flashattention面试讲解-timeline-interview-text",
      "type": "text",
      "x": 906.0,
      "y": 1438.0,
      "width": 725.0,
      "height": 150,
      "angle": 0,
      "strokeColor": "#243042",
      "backgroundColor": "transparent",
      "fillStyle": "solid",
      "strokeWidth": 1,
      "strokeStyle": "solid",
      "roughness": 1,
      "opacity": 100,
      "groupIds": [],
      "seed": 91001,
      "version": 1,
      "isDeleted": false,
      "boundElements": null,
      "updated": 1,
      "link": null,
      "locked": false,
      "text": "面试题\n1. FlashAttention 为什么降低 HBM traffic\n2. online softmax 的 m/l/O 三个状态如何保证数值正确\n3. v2 的 work partitioning 相比 v1 改善了什么",
      "fontSize": 17,
      "fontFamily": 5,
      "textAlign": "left",
      "verticalAlign": "top",
      "containerId": null,
      "originalText": "面试题\n1. FlashAttention 为什么降低 HBM traffic\n2. online softmax 的 m/l/O 三个状态如何保证数值正确\n3. v2 的 work partitioning 相比 v1 改善了什么",
      "autoResize": false,
      "lineHeight": 1.25
    }
  ],
  "appState": {
    "gridSize": null,
    "viewBackgroundColor": "#ffffff"
  },
  "files": {}
}