This project was a research project in which I embedded my private writings, questions, writing fragments, notes, and reading wishlist into a semantic space using a language model to create an emergent map of a mind. Such a platform would have many interesting applications, particularly if you shared it with others:

This was fun, but I became distracted by AI research and then the Archive and it fell in my priority list. If you'd like to pick up where I left off, DM me! This document is my stream-of-consciousness Captain's Log.

Amazing Twitter scraper: https://github.com/twintproject/twint

Contact Misha from hier topic modeling opt transport paper

Ways to build document representations:

Complete app ideas:

The goal is to reliably and quickly accumulate wisdom.

What I really want is to think about one topic at a time, from many sources (Twitter is a source too, many great thinkers aren't book writers), collect/recombine/internalize my perspectives, and integrate my conclusions into a worldview.

Observation: forgetting is default, write about what you read, articles are great too not just books, great sources accumulate constantly and perfect curriculum generation ahead of time is difficult especially if you consider articles, context switching happens at many granularities, great essays are composed of many great thought fragments, articulating them allows you to test them, great writing requires days of ruminating unless you already have a deep understanding of the topic.

Query: a few words, then find similar docs and generate reading/remembering/writing inboxes. Let these mix together for a week or two and you'll produce excellent ideas and writing.

The best thinkers on many topics have not written books. They communicate through blogs and Twitter.

Keeping abreast of this content and integrating it into durable wisdom is overwhelming.

Solutions: curriculum (books), manually tag (frictionful)

Project it all into semantic space, and only engage with one selection at a time. As you focus on that, your clippings and favorites will accumulate safely in other parts of your mind graph.

Useful platform. A few ideas:

Each dot will either be a Tweet, a book from my Goodreads, an article I clipped into Evernote, or a document I wrote in Notion. In effect my whole knowledge graph will be in here in one ZUI.

SYNTOPIC READING: focus on one piece at a time. Read about it from all angles.

WRITING: find that writing is an indispensable part of thinking. I also have to ruminate on a subject for several days to do my best writing, especially for a new topic. I make notes of thought fragments, and they're added into a writing inbox automatically.

REMEMBERING: arbitrary subsets of semantic space can be exported to Anki.

IDEATING: one generator is combining ideas. Great combinations are not too similar and not too distant - there's a Goldilocks zone that's most fertile. This corresponds to a range of distances in semantic space.

Pair two complementary topics, like writing and cognition or love and friendship, can also be generative. DOUBLE SYNTOPIC READ-IATING

IDENTIFY BLIND SPOTS: multiple projections can exist: one optimized for laying out your content, one for you and a group of friends, and another for everyone's content. if you identify a gap in your semantic space, you could find recommendations from friends who have read things in that space.

DEEPEN FRIENDSHIPS: with a more complete understanding of where your mind graphs overlap, you can give and receive better recommendations and start more and deeper conversations.

IMPROVE INTELLECTUAL DIVERSITY: find people with totally complementary knowledge.

IDENTIFY COLLABORATORS: find people with important overlaps, and connect with them as study buddies, paper reviewers, cofounders, or in relationships.

Emergent organization system of your mind. Platform with many interesting applications.

IDENTITY SHARING: I might even generate an old timey map from this, where place names correspond to dominant topic models from that area, and where mountain ranges correspond to the depth of your writing.

Compared to roam: this is a meta layer over all document apps. anything that roam can do export will work. I'm very careful not to build a note app - I've written a suite of data migration tools instead. All links open in their respective apps, and this app updates seamlessly where possible.

PROBLEMS: documents are often related in a graph, not just semantically - how to create embeddings that represent this? Just concatenate a topic model to a graph representation and project it?

Modules approach? Amazing Marvin style? Teach people to use each one?

https://github.com/IBM/HOTT/blob/master/distances.py

https://github.com/IBM/HOTT/blob/master/hott.py

https://github.com/IBM/HOTT/blob/master/knn_classifier.py

https://github.com/IBM/HOTT/blob/master/main.py

Questions:

HOTT paper description outline:

ot.emd2(p, q, C)

[method(doc, x, C) for x in X_train.T]

Word compositionality

Gensim. Try to detect bigrams and trigrams.

Oooh, I really ought to use sense2vec to preprocess this stuff, not word2vec.

I really want to do sense2vec → topic modeling → umap.

After that, might try sense2vec → topic modeling → WMD but for topic senses → umap!

For visuals: activation atlas of topic model senses. and make it a ZUI. Whoa, this will be amazing. Then: submit to ACL?

Wow: to generate

sense2vec: download other parts:

OK. My corpus is far too small to generate realistic topics. I need to build a more comprehensive topic model on a very large dataset. Wonder if I could find a pretrained one.

Wrong - I just had number of topics set to 4!

Whelp, looks like coherence score isn't everything. There's a bunch of junk in here. Hmm, they're all zero probas, too.

[]To clean these up: check plaintext of https://www.notion.so/jasonbenn/Jay-G-5901c87500d34106894bf15e50f359a1, clean out any HTML/link junk, then rerun topic modeling and see what else looks weird. James V, too.

Hmm. All these topics are pretty bad. I wonder what's happened. Probably just a ton of noise.

Will need to capture higher LDA coherence before my map makes much sense, unfortunately.

But let's proceed to sense2vec anyway.

Another thing to consider trying: TopMine.

Actually, this deserves more research.

Automated Phrase Mining from Massive Text Corpora: https://github.com/shangjingbo1226/AutoPhrase

Research Jiawei Han's publications: he's got an H-Index of 171 and works on graphs and text representations.

Goal:

Documents: should be connected semantically and by graphs. Some of them reference each other. 3m to find papers.

Enriching BERT with Knowledge Graph Embeddings for Document Classification: classifying books with cover blurbs, metadata, and knowledge graph embeddings for author information! Holy shit!

Message Passing Attention Networks for Document Understanding: we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). Includes code.

Document representations for docs - could be tweets, could be 100pg articles

Current: sense2vec → topic model (divide long articles/books to chapters? make artificial summaries using quotes?)

Alternative: anything using language models?

Seeking: long document representations?

Graph representation library DGL. Timo

Guided Similarity Separation for Image Retrieval: they used graphs to identify related images, and used those in embeddings, somehow.

Cluster documents.

UMAP straight on topic models

WMD → UMAP

Talk to about document representations:

How does UMAP/t-SNE straight on topic models look?

5m: close all tabs.

Compress outstanding questions:

https://plot.ly/javascript/zoom-events/

https://plot.ly/javascript/lasso-selection/

On right: newest content. For now, just make it last 5 entries.

Nicer hovertemplate in plotly.

Man, huge progress today.

Let's try rescraping a page - why are properties coming through like this? {')qh{': [['Yes']], "rh!'": [['Life']], 'title': [['Wisdom is mostly an aesthetic']]}

OK, they were slugified.

Read vs unread: whether it had any highlights.

Roadmap:

I want documents from Notion, Evernote, and Goodreads.

Then, I want document embeddings (weighted average BERT reps, tf-idf).

Then, document embeddings to UMAP.

Finally, a web interface to view all this.

Once I have small-scale control over my thoughts and ideas, what high-level structures feel best? Focusing on a single area for an entire month? 1 per week? Or rotating through 4, 2 days each? So many possibilities!

Well, it's interesting. Sometimes the document is made of many components (Goodreads books = author + series + book description + 3 highlights), sometimes it's just an entire document.

I think it'd be interesting to store information as closely to its original form as possible - which is different entitites that are related to each other - and then provide APIs that produce documents from these representations. And there can be many APIs for making documents. The Goodreads API set could include various combinations of sources. Notion might include or exclude the title or the backlinks. Anything is possible! All that matters is that document APIs take no arguments, read from the DB, and produce lists of strings.

So the Notion APIs are backwards: store the whole document, return them whole as documents, and let my BERT transformer transform things (or not) to Text. I might delete the whole Text concept now, actually. Well - things that can be embedded need a mixin

Shoot. Notion is tough because documents are really collections of blocks, and it's not obvious when each are updated. So I think my best bet, for now, is just to make each document a list of.

OK, I figured out how to serialize and deserialize Notion API responses to the DB - they're just JSON.

OK. Now my problem is that I would like to save JSON somewhere into the DB. This JSON can become an embeddable document, but because it could become one in multiple ways, it doesn't really make sense to make Embeddable have a JSONField. Also, other entities clearly only have one way to embed things... or do they? Maybe they don't. I'd honestly also prefer to have many embeddings. Honestly, I don't have that much text, and it would be convenient to store it all in a way that's duplicated... hundreds of Evernote notes, hundreds of Notion documents, hundreds of Goodreads books - it's really not that much. I can handle. Pocket content should go to Goodreads if I really want to savor it.

How do I elegantly access notion-py JSON things? Init into object, add a bunch of accessors? Yeah, probably. Better than passing a bunch of junk around.

I've come real far, but now I need to scrape and store all nested documents in a doc JSON - but not nested pages.

^ Example child with "content". If I fetch all this content, what are the types?

Hmm. If I flatten the content, it'll look less like its source, and eventually I'll have to rewrite all this. I won't be able to handle complicated pages with multiple columns, for example.

If I do flatten it, I simplify logic for myself in the next method, but... I'm not reflecting the original structure very well, which is a principle I'd like to stick to if possible.

What's the best way to unfold a tree structure? Build the tree with DFS? Explore each node? Then topo sort to lay them all out.

I want to go to the bathroom, get a bite, and then retry this.

What I do now: unfold and flatten the tree with BFS.

What I want to do: it's a bunch of list trees. Those will be easy to flatten once I have the whole structure.

How to do it: add nodes to visit in DFS order to a list, if I hit a string node insert its content right afterwards and continue, figure out how to build up tree structure after. Maybe pass 1 builds dictionary of content to replace,

fuck yeahhhh!! got it, and did it the right way!!

OK. Free. Now, find a relaxing spot for a couple hours, then catch the last bits of papers, then dinner at 7, then W&B drinks after, then maybe a party or home to program. What a nice night! Caffeine??? Yes.

{'id': '688cf6ee-9753-4555-9cdc-fb8d28a50e08', 'version': 55, 'type': 'text', 'properties': {'title': [['‣', [['p', 'cab1d75a-1e31-4306-8907-f8bd11a9d140']]]]}}

{'id': '30397187-ce48-4cf3-bec6-4ee065fac2e8', 'version': 16, 'type': 'page', 'properties': {'title': [['Copy of '], ['TOP LEVEL PAGE']]}, 'content': ['38b23acb-e304-4252-9915-4613f3defc98']}

Real world problem: there are many types of things. Some don't work.

Plotly docs: https://plot.ly/javascript/reference/#scatter-marker-line-colorscale

Hey, it's not looking half bad!

Random intermediate notes...

interface:

Systematically generate great ideas.

For what? I want to be wise. For what? I want to be sure I'm spending my life well!

Worldview idea: collect facts, anecdotes, make it easy to build opinions that have more or less support from facts/anecdotes

Support building idea on top of a collection of fragments, and building other, even competing ideas on top of similar fragments. Should be able to see underlying fragments too.

Graph-based approximate NN search, implemented in Rust with Python bindings:

My long-term goals: improve wisdom. To that end, I want a reading and writing queue, and a writing system. I would feel good with this morning if I get that done.

Hmm. A few branches:

Converation on creativity, how great people think:

Names: overview effect (astronauts, LSD, monks reaching nirvana), wisdom system, worldview

Features wishlist: scrape Goodreads lists & quotes, the Twitter Idea Stream, the Kindle highlights Idea Stream (extract with this?).

What do I want? In my free time, I want confidence that I am spending my time well, and growing wiser. To that end, I'd like a process that works with me.

Major obstacle now: Anki is no fun. Make those templates work.

Next obstacle: making writing a habit. Where do I write my book notes? In bed? No, it should be more serious than that. Maybe in the morning, before my shower? But then I won't be thinking about work. Perhaps that's OK on days where I'm doing straightforward things.

For now, let's just get that Anki feature working, then test it on myself and with Taylor.

Bonus goal: export into Anki automatically.

Start: 3:48pm.

Improve the Anki experience a bit.

Goal 1: backlinks between Worldview notes.

Goal 2: backlinks into People notes!!

First of all: can I detect links? If not, fork this program, or edit it directly. Use this URL: https://www.notion.so/jasonbenn/Status-is-zero-sum-but-we-can-invent-many-status-hierarchies-36c730c458dd4355837fe747baa2991f

What are the types of tables in Notion? The tables we currently support are block (via Block class and its subclasses, corresponding to different type of blocks), space (via Space class), collection (via Collection class), collection_view (via CollectionView and subclasses), and notion_user (via User class).

Nice! Found the link ID buried in the blocks with the right caret character!

Backlinks should always be text. Can I write these into another note? Text is better because it doesn't create elements in the sidebar dropdowns. It also is easier to incorporate into sentences.

This would be a nontrivial amount of work...

Holy COW I can't believe how much better cosine similarity is than a simple dot product (for NLP - image interpretability research uses dot products). Cosine similarity normalizes their lengths, right?

These suggestions are just so awesome.

Graph embeddings.

How do you surface unique things?

You could probably quantify how unique it is.

Larger context.

What do I think of "see everything all the time"?

I want supporting data, opposing data, extensions of the idea, opposing viewpoint, trends that inform it. Without filtering, all the other information would be overwhelming.

Large shaded area of everything this article touches on.

NLP will lead to totally different UI/UX:

Theories of the amount of notes I want to see:

Tweet: I'm glad you guys are digging into this, because it's the path I feel most uncertain about.

What is thought? Read Wikipedia summaries of great books on thought. Cicero, etc. Philosophy of thought?

Build argument mapping dataset! Classify as supporting/opposing.

Potential datasets for finding opposing viewpoint directions: argument mapping, rumor classification research, RumourEval, rumour stance prediction and rumour verification,

Most relevant tweet of all time, responded:

Question: when and how do I translate a thought into Twitter to get feedback from the hivemind? Do I create a boolean field? Do I write a program to automatically post them to Twitter sometimes?

Buster Benson has been building and revising his worldview incrementally since 2012! I love how he classifies them by strength and how they might be falsified.

All knowledge you wish to incorporate into your Worldview can be categorized as a Fact, a Belief, a Process, or a Question. Questions grow up into Facts and Beliefs. All can reference each other.

People must also be Strongly Connected (2+ references in my journal) to land in Anki.

What about Ideas? Is that worth elevating to a concept I think about repeatedly? I imagine flipping through my ideas deck would be really fun and inspiring. Putting them in Anki gives me more confidence that I won't forget them... otherwise building all these different processes to check various parts of my Notion would be too hard/I'd get lost.

The Idea Stream:

Make connection similarity a tuneable parameter: how similar does content have to be for it to be suggested. Could stimulate creativity.

Valid possible goals:

Possible pain points:

Beautifully designed landing page, I know exactly what it does and why I'd want it:

Core parts of my system:

Let's get the ml-box online more reliably, so that we can do 1. Move keyboard, find a mouse, get the projector hooked up. Gah. That sounds annoying. How bout: make random vectors, save those instead lol.

Work on 1.

Definitely should not feel like chores, like email. Or even FB notifs.

Should feel more like Instagram. They're rewards! And maybe they fade after a week or so. Or they come up again, later, randomly, so you don't have to worry about connecting them when they come up.

An Edges app.

Solves the problem of worrying about where to put stuff as you find it.

Roam also solves this problem: you just drop a note.

But: "the more I use Roam, the less organized I feel." (Taylor).

You promise yourself you'll organize everything, but you never do.

I wonder if revisiting them periodically with Anki would change this? Seems unlikely.

Roam's problem: enables too many connections, doesn't force you to prioritize or actually think about them. Also doesn't make you evaluate quality of source or quality of connection.

Stream of highlights since you last checked.

For each, suggest similar opinions. Hotkeys can either establish links or you can write something about that link. Can send to either Thought doc or Anki flashcard (suggest the deck!).

Great inbox processing software: gmail. Superhuman.

Great software for forwarding/making connections: Notion, Bear, etc.

Zoomable UI?

Oh snap, Zapier supports webhooks! That covers: Twitter, Evernote, Pocket.

I can export manually from: MarginNote, Otter, Kindle.

If a piece of information isn't incorporated into an opinion, a process, or my facts base, then what's the point?

Perhaps the ideal process is to just immediately suggest the connection as you're reading it. After all, that is when you have the context about that thing.

Examples of great explanations of tools for thought:

Knowledge platforms:

Trustory: so, so bad. So many uninformed opinions. No credibility whatsoever. I disagree with literally every opinion in their programming community. This is where Quora shines: subject matter experts respond to questions, and questions and answers are heavily moderated and cleaned up.

Allsides: presenting opposing viewpoints. How does it feel to read both at the same time? Confusing - don't know where to click. Nudges me towards center when sometimes facts really do support one side or another. It's pure opinion, but it should be more fact.

Debate is often framed by the phrasing of the question. Case in point: RationalWiki, which purports to offer objective, rational analysis of issues and individuals in the world of politics, business, and media, but which in reality has turned into a Marxist/leftist extremist dunkfest where they shit on anyone to the right of Bernie.

These are all trying to mix the opinions of others. I just want to collect my opinions and make sense of them all, and support/refute them as appropriate.

What we need is a boxscore for each piece of evidence. Maybe that's the layer that could go over everything. Reputation as a service. Reviews, trustworthiness. Different from reach. Relies on inventors, not influencers.

https://twitter.com/c4mer0n/status/1197683865369755648
https://changeaview.com/
https://felonvoting.procon.org/
http://www.parli.co/
https://www.qutee.com/
https://perspectroscope.com/
https://twitter.com/LogicallyHQ
https://changeaview.com/
https://twitter.com/balajis/status/1197696369210904576
https://en.wikipedia.org/wiki/Knowledge_Graph

Research interests

Collaborative filtering

You could prime it with hints

Goal features

When I stake out an opinion on ML, relevant papers I've saved surface. I can toss papers into my system and know they'll come later when I'm most curious about them. (Scrape abstracts?)

Understands the relationship between papers and people (entity resolution).

Goal tonight: get Goodreads into my DB.

Design!

SF Symbols

Material design icons

Observation: it's really exciting to hear a great idea and put it into my system. I have confidence that I'll think about it more, even if I don't have time right now. I don't have to be preoccupied with forgetting it. I think this system enables me to search more widely for ideas, in the same way that Anki enabled wider learning.

Interface:

Everything gets

What is next? What is the next goal that excites me?

Having a DB full of Goodreads highlights. For now, just have highlights have nullable FKs to Book or NotionDoc.

Next

Most important next thing

I want to see things I've already read - Goodreads read and unread - show up in the map.

I've got a solution - what's the problem?

I forget everything I read, and they mostly don't impact what I believe.

I want a tool to help me understand how all my knowledge fits together.

Main app: put everything into massive DB, documents in S3, compress them with BERT, map them with UMAP.

Uses: reading recommendations, explore the fog of war, find writing you should link, etc. Queryable. Cosine similarity happens via procedure.

MVP:

All reading is broken. I read things and barely remember a single fact a week later. The only fix is to explain or talk about the things you read, but most content isn't like that.

Solution is have all read and written content mapped, such that similar content is next to each other.

Next project: connection suggestions!

Notion-to-text program: for any Notion page, convert all contents to one string.

MVP vectorizer: just use sklearn's TF-IDF, see how it goes.

DB: sqlite. Load everything, do similarity search in memory. Will be a while before I have many thousands of documents and their vectors won't fit in memory.

MVP: copy-paste text into inputs.

Should also work for Notion ←→ Notion connections. Can either pass raw text or a Notion link.

Goal: to build an insight machine: that means integrating content that I've saved and written.

Ben recommends GPU ML algorithms, and newspaper3k for parsing.

Worldview metrics (add to a dashboard!)

Maybe these should go to separate decks?

Improving my UMAP projection

idea: the distant future?

Inputs:

Record more conversations with Otter.

Evernote collects my readings and highlights.

Classifiers:

A program identifies the information worth saving, and/or its type - is it about a person, an idea or opinion, a great question, or a joke for my journal? Fact v opinion v experience classifier.

Another program suggests which Notion content it should be merged into. Language model activations of each new piece of content · language model activations (in DB) of each worldview and/or great question. No similarities? Great - prompt to create a new note.

Attention:

Things to think about more are included into Anki: open questions, my friends and acquaintances.

Content to read is suggested based on opinions that need fleshing out.

Outcomes:

High trajectory: the next time we have a conversation, there's a good chance I've internalized your insights and further developed them

Sample efficiency: nothing worthwhile is forgotten

v1?

Done

Are there other great streams/sources of information that aren't Anki? Suggestions based on your writing?

Main goals:

Opinions as note titles is a breakthrough. Forces me to defend it, articulate it, explain it.

Opinions as Anki cards is slightly disappointing.

What I want is to combine this with my reading list. Building a worldview is to read widely and critically.

Related notes from Evernote could be inserted at the bottom of notes. Test the MVP. Implementation would copy content from Notion into Evernote, then use the Evernote suggestion API, which can take any number of notes.

Implications: to get content to appear in my reading list, it has to first be motivated by an opinion.

Related notes from Notion as well.

Outstanding questions:

Compare representations to find the direction.