TL;DR: My iPhone 16 Pro Max produces garbage output when running MLX LLMs. An iPhone 15 Pro runs the same code perfectly. A MacBook Pro also runs the same code perfectly. The tensor outputs on the 16 show numerical values an order of magnitude wrong. I suspect it points
  • panda_abyss@lemmy.caEnglish
    172·
    1 month ago

    Yea, but those are all using heaps of proprietary heuristics.

    The beauty of LLMs and one of their most useful tasks is taking unstructured natural language content and converting it into structured machine readable content.

    The core transformer architecture was original designed for translation, and this is basically just a subset of translation.

    This is basically an optimal use case for LLMs.

    • MolochHorridus@lemmy.mlEnglish
      72·
      1 month ago

      Quite obviously not the optimal use case. “The tensor outputs on the 16 show numerical values an order of magnitude wrong.”