TL;DR: My iPhone 16 Pro Max produces garbage output when running MLX LLMs. An iPhone 15 Pro runs the same code perfectly. A MacBook Pro also runs the same code perfectly. The tensor outputs on the 16 show numerical values an order of magnitude wrong. I suspect it points
      • panda_abyss@lemmy.caEnglish
        172·
        1 month ago

        Yea, but those are all using heaps of proprietary heuristics.

        The beauty of LLMs and one of their most useful tasks is taking unstructured natural language content and converting it into structured machine readable content.

        The core transformer architecture was original designed for translation, and this is basically just a subset of translation.

        This is basically an optimal use case for LLMs.

        • MolochHorridus@lemmy.mlEnglish
          72·
          1 month ago

          Quite obviously not the optimal use case. “The tensor outputs on the 16 show numerical values an order of magnitude wrong.”