<aside> 📝

This is an old version of the website. To see the new one go to:

CognitionTo

</aside>

<aside> 📢 TLDR: This page is resources of youtube, research papers, books, blogs, discord, tools, glossary, friends, and more.

</aside>

📝 Blog

Untitled

📝 Contact

Untitled

💭 Overall thoughts

Over the last couple of months I’ve been doing some cool stuff. Personally I think, this it the most most interesting and important work I’ve ever done. I was talking to a couple of friends in my study group and it dawned on me that I should compile my notes I’ve been taking into a single living document that I can update overtime as I build and learn new things.

I’ve been working as a software engineer for ~5 years but there’s always been something interesting about machine learning. I don’t think there’s another field that I know of where implications at the lowest level have such a huge implication on how society could be shaped.

I’ve tried to reference as much material as possible, If I’ve missed any references, made any mistakes, or if you want to get in touch add me:

• Contact:
  • github, linkedin, 🤗, **x,** [email protected], discord: joshcarp, book a meeting

Unlike my diet, everything on this page is 100% organic. Anything generated with AI will be indicated with 🦾.

📝 Research Papers

Bellow is a list of the interesting papers i’ve read, or at least the ones I’ve bothered to keep notes on. Note: Diagrams have been generated by Claude 3.5 Sonnet and haven’t been manually edited by me (yet).

The OG papers

Foundational papers on the current LLM paradigm. Attention is all you need introduced the Transformer, Improving Language Understanding by Generative Pre-Training introduced the GPT.

%%{init: {'theme': 'base', 'themeVariables': { 'nodeTextColor': '#333333', 'mainBkg': '#f0f0f0', 'lineColor': '#F8B229'}}}%%

graph TD
    subgraph "The OG papers"
        A["<a href='<https://arxiv.org/abs/1706.03762v7>'>Attention Is All You Need</a>"]
        B["<a href='<https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf>'>Improving Language Understanding</a>"]
        A --> B
    end

• **Attention Is All You Need -** The OG introduced the idea of self-attention and the encoder/decoder architecture for language translation tasks (the encoder later got dropped because it was only used for translation). Another breakthrough from this paper was the training; “The Transformer allows for significantly more parallelisation and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.” - This fact here was what let it: overtake RNNs (which weren’t parallelisable), and lead NVIDIA to be worth more than 2.7 Trillion token credits.