Für einen Test von deepseek-r1:70b in der Q4_K_M Variante habe ich mal eine SimpleBench Frage genommen und sie dem deepseek-r1:70b Modell in der Q4_K_M Variante gestellt.
SimpleBench (simple-bench.com) ist ein Multiple-Choice-Text-Benchmark, der entwickelt wurde, um die Fähigkeiten von großen Sprachmodellen (LLMs) in Bereichen wie räumlich-zeitlichem Denken, sozialer Intelligenz und linguistischer Robustheit gegenüber sogenannten „Trickfragen“ zu evaluieren. Der Datensatz umfasst über 200 Fragen und ist öffentlich zugänglich. Er kann verwendet werden, um LLMs zu testen und deren Leistung mit bestehenden Modellen zu vergleichen. Ich habe mal die Frage 8 gewählt und auf deutsch übersetzt.
SimpleBench Frage 8 (deutsche Übersetzung)
In einem Regal stehen nur ein grüner Apfel, eine rote Birne und ein rosa Pfirsich. Das sind auch die jeweiligen Farben der Schals von drei unruhigen Schülern im Raum. Dann wird eine gelbe Banane unter den rosa Pfirsich gelegt, während eine lila Pflaume auf den rosa Pfirsich gelegt wird. Der Junge mit dem roten Schal isst die rote Birne, der Junge mit dem grünen Schal isst den grünen Apfel und drei andere Früchte, und der Junge mit dem rosa Schal wird [ _ ].
A. nur die gelbe Banane essen
B. die rosa, gelben und violetten Früchte essen
C. nur die lila Pflaume essen
D. den rosa Pfirsich essen
E. zwei Früchte essen
F. keine Früchte essen
Deepseek-r1:70b auf dem Jetson Orin 64GB
Für den Test habe ich das Model in ollama geladen und in Open-WebUI ausgewählt. Im Hintergrund sieht man ein Grafana Dahboard, in dem ich ein paar wichtige Kennziffern wie Temperatur, GPU Performance, Energieverbrauch und RAM Nutzung beobachte.
Der Test dauert relativ lange, da das 70B Modell den Jetson Orin schon ein wenig an seine Grenzen bringt. Dann ist das Ergebnis aber korrekt (Antwort F).
Zusammenfassend kann man festhalten, dass der Jetson Orin in der Lage ist, 4 Bit quantifizierte Modelle laufen zu lassen, auch wenn nur wenige Token pro Sekunde erreicht werden. Das Zusammenspiel von Ollama, Deepseek-R1 70B und Open-WenUI funktioniert wirklich beeindruckend gut.
Hier noch mal die komplette „Thinking“ Ausgabe die das Model erzeugt hat.
Thought for 11 Minuten
Okay, so I’m trying to solve this logic puzzle about the fruits and the students with different colored scarves. Let me break it down step by step.
First, there’s a shelf with three items: a green apple, a red pear, and a pink peach. Each of these corresponds to the colors of the scarves worn by three restless students in the room. So, we have a boy with a green scarf, one with a red scarf, and another with a pink scarf.
Then, some changes happen:
- A yellow banana is placed under the pink peach.
- A purple plum is placed on top of the pink peach.
So now, the arrangement on the shelf must be something like this from bottom to top: yellow banana, pink peach, purple plum. Below that, there’s still the green apple and red pear, right? Or maybe the banana is added under the peach, so the order would be banana at the bottom, then peach, then plum on top of the peach. So the shelf now has five items: apple (green), pear (red), banana (yellow), peach (pink), and plum (purple). But wait, initially, there were only three—apple, pear, peach. Then they added banana under peach and plum on top of peach. So maybe it’s structured as banana (under peach), then peach, then plum on top of peach. So the shelf would have apple, pear, banana, peach, and plum.
Wait, no. Let me think again. The initial setup is only a green apple, red pear, and pink peach. Then, they add a yellow banana under the pink peach. That means the banana is placed below where the peach was, so now it’s banana at the bottom, then peach on top of that? Or maybe the shelf isn’t stacked but arranged in some order. Perhaps the original three are on the shelf: apple, pear, peach. Then they add a banana under the peach—so either next to or beneath it. And then place a plum on top of the peach.
But perhaps it’s better to think that after adding the banana and plum, there are five fruits in total: green apple, red pear, yellow banana, pink peach, purple plum.
Then, each boy eats certain fruits based on their scarf color:
- The boy with the red scarf eats the red pear.
- The boy with the green scarf eats the green apple and three other fruits.
- So that leaves the boy with the pink scarf to eat… something. But what’s left?
Wait, let me list all the fruits after the additions: green apple (eaten by green scarf), red pear (eaten by red scarf), yellow banana, pink peach, purple plum.
So the green scarf eats the green apple and three other fruits. Let’s count:
Green scarf eats:
- Green apple
- Yellow banana
- Pink peach
- Purple plum
Wait, that would mean he eats four fruits in total: his own plus three others. But let me check again.
The problem says, „der Junge mit dem grünen Schal isst den grünen Apfel und drei andere Früchte,“ which translates to „the boy with the green scarf eats the green apple and three other fruits.“ So that’s four fruits in total for him: green apple + 3 others.
So from the five fruits, red pear is eaten by the red scarf, green scarf eats green apple plus three others. That would mean the green scarf eats four fruits, leaving only one fruit for someone else, but wait, there are only three boys: red, green, and pink scarves.
Wait no—initially, it’s just those three students. But then in the problem statement, after adding the banana and plum, we have five fruits, right? Green apple, red pear, yellow banana, pink peach, purple plum.
Now, each student eats based on their scarf color:
- Red scarf: eats red pear (1 fruit)
- Green scarf: eats green apple + 3 others (so four fruits total)
- Pink scarf: ?
Wait, that’s a problem because there are five fruits in total. If red eats one, green eats four, then pink would have none left. But that can’t be right because the question is about what the pink scarf does.
Alternatively, maybe I’m miscounting. Let me recount:
Fruits after additions: 5 (apple, pear, banana, peach, plum).
Red scarf eats pear (1), green scarf eats apple and three others—so he would eat four fruits total. That leaves only one fruit for the pink scarf? But that’s impossible because if green scarf eats four, red eats one, then all five are eaten.
Wait no—the question says „der Junge mit dem grünen Schal isst den grünen Apfel und drei andere Früchte.“ So he eats the green apple plus three others—so four fruits. Then, red eats pear (1), and pink would have none left because 4 + 1 = 5.
But that can’t be right because then option F says he eats nothing. However, let me check if I’m misunderstanding.
Wait, maybe the initial shelf had only three: apple, pear, peach. Then they add banana under peach and plum on top of peach—so perhaps the arrangement is banana at bottom, peach in middle, plum on top. So total fruits are apple (green), pear (red), banana (yellow), peach (pink), plum (purple).
Now, red eats pear, green eats apple plus three others—so he would eat four fruits: apple, banana, peach, and plum. That leaves nothing for pink. But that can’t be because there’s the option where pink eats only peach or something else.
Wait, maybe I’m misinterpreting „drei andere Früchte.“ Does it mean three other in addition to his own? So green eats 1 (apple) + 3 others = 4 fruits. Then red eats pear (1), so total eaten is 5, leaving nothing for pink. Thus, pink would eat nothing, which is option F.
But let me think again—maybe the „drei andere“ are in addition to his own, making it four total. So green scarf: apple + banana, peach, plum = four. Red eats pear. That’s five fruits. Pink has none left. So the answer would be F: keine Früchte essen.
But wait, let me make sure I’m not missing anything. Maybe the shelf is structured so that when they add the banana under the peach and plum on top, perhaps only certain fruits are accessible or belong to each boy.
Alternatively, maybe after adding, the arrangement is such that each boy can only reach certain fruits based on their position relative to their scarf color’s fruit. But I’m not sure.
Wait, no—the problem doesn’t specify any restrictions on which fruits they can eat beyond their own color. So green eats apple and three others—so four total. Red eats pear—1. Pink has none left because 4 + 1 =5, so pink eats nothing. Therefore, the answer is F.
Der Junge mit dem rosa Schal wird keine Früchte essen .
F. keine Früchte essen