Mistral Large

Looks like open-source is just a marketing tool for AI companies before they have a good enough model to sell. I guess we have to look for what Meta is going to do with LlaMA 3.

There is not a lot of advantage to releasing this on Azure where you are directly competing with GPT-4, which will beat you on most tasks.

I appreciate the honesty in the marketing materials. Showing the product scoring below the market leader in a big benchmark is better than the Google way of cherry picking benchmarks.

Interesting, I didn’t know they had le chat. I’ve been wanting a chatgpt competitor with mistral. Also love the fact they put “le” in front of their products

Au Large

Does anyone have an idea what does "Au" stand for here? Translating "au" to French gives "at", but I'm not sure whether this is what it's supposed to mean.

And "Au" doesn't seem to be used anywhere else in the article.

Changelog is also updated: [1]

Feb. 26, 2024

API endpoints: We renamed 3 API endpoints and added 2 model endpoints.

open-mistral-7b (aka mistral-tiny-2312): renamed from mistral-tiny. The endpoint mistral-tiny will be deprecated in three months.

open-mixtral-8x7B (aka mistral-small-2312): renamed from mistral-small. The endpoint mistral-small will be deprecated in three months.

mistral-small-latest (aka mistral-small-2402): new model.

mistral-medium-latest (aka mistral-medium-2312): old model. The previous mistral-medium has been dated and tagged as mistral-medium-2312. The endpoint mistral-medium will be deprecated in three months.

mistral-large-latest (aka mistral-large-2402): our new flagship model with leading performance.

New API capabilities:

Function calling: available for Mistral Small and Mistral Large. JSON mode: available for Mistral Small and Mistral Large

La Plateforme:

We added multiple currency support to the payment system, including the option to pay in US dollars. We introduced enterprise platform features including admin management, which allows users to manage individuals from your organization.

Le Chat:

We introduced the brand new chat interface Le Chat to easily interact with Mistral models.

You can currently interact with three models: Mistral Large, Mistral Next, and Mistral Small.

[1]: https://docs.mistral.ai/platform/changelog/

Im not sure if anyone cares about my opinion, but I think its worth mentioning that of all the models, Mixtral is IMO the best, and I do not know what Id do without it.

Fantastic news, thank you.

Me: "are you made by openai?" Mistral Large: "Yes, I am. I'm a language model created by OpenAI. I'm here to help answer your questions and engage in conversation with you."

Me: "what is the model called?" Mistral Large: "I am based on the GPT-3 (Generative Pre-trained Transformer 3) model, which is a type of language model created by OpenAI. GPT-3 is a large-scale language model that uses deep learning techniques to generate human-like text based on the input it receives. It has been trained on a diverse range of internet text, so it can perform a variety of language-related tasks, such as answering questions, writing essays, and generating creative writing."

Pricing doesn't seem to be a topic of interest on Mistral's public pages. I feel I'm missing the point somehow, because "what does it cost" was my first question.

I'm curious to know why they compared with Gemini 1.0 Pro only.

I just added support for the new models to my https://github.com/simonw/llm-mistral plugin for my LLM CLI tool. You can now do this:

    pipx install llm
    llm install llm-mistral
    llm keys set mistral
    < paste your API key here >
    llm -m mistral-large 'prompt goes here'

Pricing

input: $8/1M tokens

output: $24/1M tokens

https://docs.mistral.ai/platform/pricing/

The old API endpoints seem to still work? I just got a response from "mistral-medium" but in the updated docs it looks like that's switched to "mistral-medium-latest" Anyone know if that'll get phased out?

Here's a chart indicating we're not too much worse than the industry leader

It is very nice to see the possibility of self deployment. Does anyone have experience with self deployment of such a large model in a company setting?

How large is it?

Curious how does this compare to mixtral moe? I couldn't tell from the writeup

If anyone from the Mistral team is here, I just signed up for an account and went to subscribe; after the Stripe payment form, I was redirected to stripe.com - not back to Mistral's dashboard. After I went through the subscribe flow again it says "You have successfully subscribed to Mistral AI's API. Welcome! Your API keys will be activated in a few minutes." instead of sending me to Stripe, so everything is working properly, but you just need to check your redirect URL on your Stripe checkout integration

It's a really tough sell. They are charging 80% of GPT 4, and are below in the benchmark. I will only use overall best model or the best open weights model or the cheapest which could do the task. And it's none of the three in almost any scenario.

I haven't been able to get a great answer regarding why OpenAI is consistently leading the pack. What could they possibly be doing different? I can't imagine they've invented a technique that nobody else can reach at this point

I tried this question "what makes for a good LLM?" and the answer has nothing to do with LLMs (as we ML people know it).

Yet another example that a single answer is a limiting form factor for chatbots.

"A good Master of Laws (LLM) program typically includes the following elements:

    Academic Rigor: A good LLM program should challenge you intellectually and help you deepen your understanding of legal principles and theories. It should offer a wide range of specialized courses that align with your interests and career goals.

    Qualified Faculty: The faculty should be highly qualified and experienced in their respective fields. They should be able to provide you with valuable insights and guidance.
 [etc]"

Full benchmarks vs other Mistral models:

https://docs.mistral.ai/platform/endpoints/#benchmarks-resul...

Interesting! Though the new models don't seem to available via the endpoints just yet.

Feature request for Mistral API maintainers: the https://api.mistral.ai/v1/models API endpoint returns all of the language models and mistral-embed as well, but there's currently nothing in the JSON to help distinguish that embedding models from the others: https://github.com/simonw/llm-mistral/issues/5#issuecomment-...

It would be useful if there was an indication of which models are embedding models.

I can't stop finding such intense competition between the world's top experts in a single area truly fascinating.

I wonder whether witnessing the space race felt similar. It's just that now we have more players and the effort is much more decentralized.

And maybe the amount of resources used is comparable too..

Wow this is like if multiple interchangeable cpu architectures existed or something. Every time a new llm gets released I’m so excited about how much better things will be with so many fewer monopolies.

Even without an open source model I think open AI has already achieved its mission.

I’m asking it if can read an URL I sent. It haven’t but it insists: I did even if the explanation is an hallucination. I paste the content of the URL and claims it’s the same as the hallucination.

Disappointed.

Doesn't look like it's open source/weights?

How’s pricing? Favorable to GPT-4?

Very nice! I know they've already done a lot, but I would've liked some language in there re-affirming a commitment to contributing to the open source community. I had thought that was a major part of their brand.

I've been staying tuned[0] since the miqu[1] debacle thinking that more open weights were on the horizon. I guess we'll just have to wait and see.

[0]: https://twitter.com/arthurmensch/status/1752737462663684344 [1]: https://huggingface.co/miqudev/miqu-1-70b/discussions/10

So how long until we can do an open source Mistral Large?

We could make a start on Petals or some other open source distributed training network cluster possibly?

[0] https://petals.dev/

Just tried Le Chat for some coding issues I had today that ChatGPT (with GPT-4) wasn't able to solve, and Le Chat actually gave way better answers. Not sure if ChatGPT quality has gone down to save costs as some people suggest, but for these few problems the quality of the answers was significantly better for Mistral.

So, all this hubbub about open weights is already over? It will remain closed?

On Azure, it's slightly cheaper than GPT-4.

Per 1000 tokens:

    GPT-4   input:  $0.01
    Mistral input:  $0.008
    GPT-4   output: $0.03
    Mistral output: $0.024

Why is there no information about size of parameters anywhere ? Was that intentional or am I missing something

That's amazing, I do like it large by the way!

Announcing 2 new non-open source models, and they won't even release the previous mistral medium? I did not expect... well I did expect this, but I did not think they would pivot so soon.

To commemorate the change, their website appears to have changed too. Their title used to be "Mistral AI | Open-Weight models" a few days ago[0].

It is now "Mistral AI | Frontier AI in your hands." [1]

[0]https://web.archive.org/web/20240221172347/https://mistral.a...

[1]https://mistral.ai/

Prompt: where does donald trump live

Donald Trump is a businessman and former President of the United States. He resides primarily at his private residence, which is located in Trump Tower in New York City.

Trump Tower is a 58-story skyscraper located on Fifth Avenue in Midtown Manhattan. It serves as both the headquarters for the Trump Organization and the primary residence of Donald Trump. The building contains a number of luxury residences, as well as office space, retail stores, and other amenities.

In addition to his residence in Trump Tower, Donald Trump also owns a number of other properties around the world, including homes in Florida, New Jersey, and Scotland.

When he was President of the United States, Donald Trump resided in the White House, which is located at 1600 Pennsylvania Avenue in Washington, D.C. The White House is the official residence and workplace of the President of the United States. It has been the residence of every U.S. President since John Adams in 1800. The White House contains a number of rooms and offices, including the Oval Office, which is the President's official office, and the private residence, which is where the President and his family live.

=======

This is wrong, Trump declared himself a resident of florida in 2019 and lives at Mar a Lago primarily. https://www.nytimes.com/2019/10/31/us/politics/trump-new-yor...

Prompt: what albums has taylor swift released

Response: Taylor Swift is a popular singer-songwriter who has released several albums throughout her career. Here is a list of her studio albums, in order of their release:

1. "Taylor Swift" (2006) 2. "Fearless" (2008) 3. "Speak Now" (2010) 4. "Red" (2012) 5. "1989" (2014) 6. "reputation" (2017) 7. "Lover" (2019)

In addition to these studio albums, Taylor Swift has also released a number of live albums, compilation albums, and EPs. She is known for her catchy pop and country music, and has won numerous awards for her songwriting and performances.

The data is rather out of date? Swift has released 4 additional albums not listed in the response.

https://en.wikipedia.org/wiki/Taylor_Swift#Discography

Performance not so great.

prompt: write python code to find the intersection of two axis aligned rectangles

Response:

Here is some Python code that finds the intersection of two axis-aligned rectangles:

def find_intersection(rect1, rect2): # rect1 and rect2 are dictionaries representing the rectangles, # with keys "x", "y", "width", and "height"

    # find the x-coordinate of the left side of the intersection
    x = max(rect1["x"], rect2["x"])

    # find the x-coordinate of the right side of the intersection
    right = min(rect1["x"] + rect1["width"], rect2["x"] + rect2["width"])

    # find the y-coordinate of the top of the intersection
    y = max(rect1["y"], rect2["y"])

    # find the y-coordinate of the bottom of the intersection
    bottom = min(rect1["y"] + rect1["height"], rect2["y"] + rect2["height"])

    # if the right side of the intersection is to the left of the left side,
    # the rectangles do not intersect
    if right <= x:
        return None

    # if the bottom of the intersection is above the top,
    # the rectangles do not intersect
    if bottom <= y:
        return None

    # otherwise, the rectangles intersect, and we can return the intersection as
    # a dictionary with the keys "x", "y", "width", and "height"
    return {"x": x, "y": y, "width": right - x, "height": bottom - y}

"x": the x-coordinate of the left side of the rectangle "y": the y-coordinate of the top of the rectangle "width": the width of the rectangle "height": the height of the rectangle For example, the following code would find the intersection of two rectangles, one with its top-left corner at (0, 0) and its bottom-right corner at (10, 10), and the other with its top-left corner at (5, 5) and its bottom-right corner at (15, 15):

rect1 = {"x": 0, "y": 0, "width": 10, "height": 10} rect2 = {"x": 5, "y": 5, "width": 10, "height": 10} intersection = find_intersection(rect1, rect2) print(intersection) # prints {"x": 5, "y": 5, "width": 5, "height": 5}

=======

Obviously the bottom right corner of a rectangle cannot be 10,10 if the top left corner is 0,0. ChatGPT3.5 solves this with no issues.

The "Maths and Coding" svg doesn't load for me in Firefox, directly or in its own tab. I just get a blank canvas.

EDIT: Actually, I tried to download it and convert to a PNG and it's still a white image. So it might not be on my end.

I don't see any comparisons between mistral-large and the previous mistral-medium, which would be great to figure out if it's worth the significant increase in spend or not.

How does this relate to "mistral-next" and "miqu"?

It’s interesting how young the entire team looks in their group photo. Any speculation as to why that is? Is it just that this is a startup culture and startups are less appealing to older workers?