Bits and Paradoxes - 20
Innovation and Imitation
Disclaimer: This post is very long. If you find it truncated by your email client, please directly visit the web version. (Or better, save it to read later?)
Hi folks,
A couple of weeks back, I decided to do a very naive text-to-speech related project. The concept is simple —a script (at least for now) that takes in any URL and reads the main article in it. I ended up doing a lot of research and tinkered with different deep learning models — this, this, and this. Eventually, I created a basic pipeline using Tachotron2 with two main components: Article extractor and Text-To-Speech (TTS)
I'd say it's fine for time being. But, it seems to be way more complicated. Partly because the TTS component has a character limit (truncates a long text). And partly because I haven't been able to use multiple voices.
This has put me on a thinking loop of imitation and innovation because this project isn't any new. There are a lot of tools that can do it. However, they haven't been able to fulfill my needs of listening to long-form essays (stemming from my podcast-listening habit).
Let's take pocket. As its long-standing user (from 2013 and onward), I am very unsatisfied with its TTS component. Its audio output is of low quality, unempathetic and almost robotic. It doesn't work in the web version either, which is kinda bummer as I rarely use my not-so-smart-phone. Although Mozilla has introduced a better Deep-Learning-based model, it still sucks.
So, the project started out of frustration as nothing has satisfied me. Now, what I have realized is that thinking in terms of a problem-solution mindset is just the tip of the iceberg. Market demands and value propositions matter a lot. There are certainly a lot of good paid services which people are using. Even if I develop this tool, it will surely fail for a number of reasons:
First, the data for re-training the model on another voice should be large (almost 3 hours of audio with a lot of tonal variation)
Secondly, re-training might take a toll on compute resources, and thus my spare time too. I'd have to constantly tune the model to get a more natural voice. It might hamper my other works.
Lastly, I barely have the knowledge to make it scalable since streaming data like audio/video is an engineering challenge.
[For now, I let it be.]
[[Innovation and Imitation]]
(Figure: Copy-Transform-Create flowchart)
Certainly, imitation when done right (properly) creates real value. It has been done for a long time now, especially with the technological advancements we have made so far.
Let’s take the iPhone as a case study.
The iPhone was a revolutionary device. It wasn't new though. Mobile phones had existed long before it was introduced. However, it completely changed the smartphone market through a touch-only interface and a slim (portable) design. Again, the multi-touch technology did exist before that. They simply innovated on top of that. On that note, resistive touch (old technology) had been in existence since the 70/80s.
There are other innovative imitations on the iPhone such as the touch-based QWERTY keyboard which took inspiration from blackberry. Because their vision was to build a touch-only "smart" phone, they ditched the idea of a physical keyboard. The "innovation" part of the touch keyboard was that they were able to add in text completion to the software, plus figure out a way to make relevant keys bigger as you type. This made typing way easier.
Now if we zoom further back in the 60s-70s, Douglas Engelbart and the research team had laid the foundations for modern tools like mouse, video conferencing, hyperlinks, word processors, and real-time collaborative editing (Re: The mother of all demos). His lab hosted one of the 3 centers of ARPANET, which is considered to be one of the key ancestors for the internet. So basically he helped invent the internet. (It's kinda cool but nobody really knows about it.)
So, what?
If you just think about one product you are using right now, I am sure there are a ton of alternatives. Say, note-taking (writing?) tools. There are a lot of them. Evernote, Google Keep, Google Doc, Notion, TiddlyWiki, Roam Research, Obsidian…you name it… Although they do similar things, each has its own set of goals for the user.
Notion markets itself as "All-in-one workspace. One tool for your whole team. Write, plan, and get organized."
Roam's motto is: "A note-taking tool for networked thought. As easy to use as a document. As powerful as a graph database. Roam helps you organize your research for the long haul."
Google Keep is a minimal application for notes and lists.
"Obsidian is a powerful knowledge base that works on top of a local folder of plain text Markdown files."
Evernote is an old elephant: “Simplify your life. Evernote is the home for everything you need to remember, and everything you want to achieve.”
Out of these, I find Roam as a strong innovation. Although superficially it seems like a usual personal Knowledge Management tool, it's a tool to map your mind, to build your [[Second Brain]] seamlessly. What strikes Roam as “different” is its treatment of blocks (think of bullet points) as a fundamental unit of knowledge that can be linked anywhere within the system. For me, that's kinda innovation, not just simple imitation. I have touched similar sentiment on issue 05.
Whenever something starts to take the market, especially with a sudden growth, it's sure to get a lot of competition. For Roam, the likes of Obsidian, Logseq, Athens have jumped in. It's sure that only a few can survive with real innovation that's a mix of imitation and originality. Even if everything co-exists, there should be a “differentiating factor” that lets each of them standout from each other which is crucial for imitation.
Innovative Imitation
Reading | Theodore Levitt | 23 min
There are a few key concepts I loved in this essay, especially about reverse R&D and probability estimates during imitation. But before that, let's try to define what innovation and imitation mean.
What is innovation?
I think the answer is fuzzy and depends on the context. But following two perspectives can help define something as "innovative"
If something is entirely new (unprecedented), it can be thought of as an innovation.
If something has been done before in some another domain, but now it is introduced in an entirely new one, it is an innovation.
What is imitation?
I think it's something which is a poor replica of the original. "Poor" in a sense, it's inefficient, barely gets the job done. For instance, during the mobile phone race in the 2000s, there were a lot of good innovations such as the Nokia n-70, Motorola's slim phones, iPhone, Android… However once the iPhone and the Android started to win the smartphone race, Nokia struggled to keep up. Eventually, their poor imitation for OS and the software became their downfall.
Imitation is endemic. Innovation is scarce.
Imitation is more prevalent than innovation because innovation needs more resources (time, money, and planning). Investing in an original idea is difficult for that very reason. You'd have to plan meticulously and do extensive R&D. And because the product (innovation) is new to the market, its success is rather unpredictable in general.
So, companies are hesitant to immediately jump into the market. There's also an equal risk of going too early because people would not be able to comprehend the newness.
Reverse R & D
Although imitation is done to minimize risk (risk aversion), to get caught in the product market and the competition, it doesn't necessarily mean it's not difficult. You'd have to breakdown the original innovation and build upon it. It's more like a Reverse R&D (reverse engineering). You'd have to imitate purposefully and strategically. Because otherwise, you'd be too late in the game.
Probability Estimates
If a new product hits a market, there isn't any competition initially because competitors are skeptical about it and fail to predict its success. By the time they realize that the product is taking up a big chunk of the market, it will be too late.
The reason we must say that Company X gave a zero probability to the product’s chances of success even in Year 3, despite the obviously worried reaction of “George, we’d better take a closer look at this,” is that no steps were taken to begin to tackle the most complex and lengthy job associated with imitation—reverse R & D.
Estimating the success rate of innovation plays a crucial role to plan the competition. The attitude of doubt and tentativeness can and should be translated into sound business practice. For that, it is relevant to observe (monitor) closely during the first few months (or an entire year). During that period, initial market research along with reverse R&D might be a viable step. Each year there could be some estimates of failure/success for the original product.
In all, clear and careful planning of innovative imitation is similar to that of innovation itself.
There are times where an innovative imitation succeeds better than the original one because, by the time you are working on it, you'd have already known "what worked and didn't work" from the original innovation. That way, with reverse-engineering the original idea (and product), you could come up with a good strategy to fix the failures.
For example, let’s take Fortnite. It found its way at the top only after when PUBG was holding the battle royale market. Although PUBG failed to fix and optimize itself, Fortnite has been doing great. (Oh! Let’s not talk about Epic’s current dispute with Apple.)
"You don’t have to get the first bite on the apple to make out. The second or third juicy bite is good enough. Just be careful not to get the tenth skimpy one."
If we take Docsumo (where I am doing active research at the intersection of computer vision, NLP, and algorithms) as another case study, what we are doing isn’t new. The product market for information extraction is pretty saturated and competitive. However, what we excel at (pun intended) is optimizing cost and speed. We provide our service at a cheaper rate, and our algorithms are very fast. That’s the “standout factor”.
Innovation isn't about big ideas but tweaking the existing one to improve upon it.
“Good artists copy. Great artists steal”
This, once again, has “everything is a remix” narrative. Copy. Transform. Create.
Principles of Effective Research
Reading | Michael Nielsen | 35 min
This is one of the profound essays on approaching research in general.
A few key points I really loved include why we fail to be proactive and how self-development and the creative process should go hand in hand.
Proactivity
You should be proactive towards your work with a strong sense of responsibility. At least in theory. It's easier said than done for a number of reasons:
You blame the external circumstances for your ineffectiveness. (Say, you keep on complaining about not having enough funding.)
Displacement Activities
You are too much caught up in smaller, less valuable activities. As a result, you aren't able to focus deeply on the actual research work. Such displacement activities may provide short-term fulfillment. But in the long run, they might be wasting your time.
Negative Internal Locus of Control
Having too much of a negative mindset is probably not healthy ( joke’s on me :/ ). This includes worrying too much about the work, feeling bad for not overcoming difficulties, constantly thinking that the work you are doing will not succeed, and such.
One powerful way to get out of that rut is to be inspired by examples of proactive people, which can happen through direct communication or reading/learning processes.
Self-development and Creative Process
While being proactive is one of the principles of personal behaviour that makes you efficient, self-development and creative process are important aspects for you to succeed in research works. Balancing both of them is crucial.
If you are too much focused on self-development and aren’t contributing significantly to the community (or wider audience), the research is pretty ineffective.
Conversely, if you are putting efforts only on publishing more and more (low quality) papers without self-development, the research is ineffective.
While self-development includes strengthening your research interests and your environment, the creative process defines what type of researcher you are going to be — problem-solver or problem-creator.
Strengthen your interests
You can strengthen your research interests by having clarity and forward-momentum on what you are trying to learn. Or defining your short-term/long-term goals. To have that clarity, you should make your environment accountable for your work.
For instance, being engaged in the community, student-teacher relationship, or discussing the work with your colleagues creates a sense of responsibility and accountability you can’t avoid.
If your environment makes you lethargic and stressed out, it's better to refactor it because motivation matters a lot than anything else.
Another path to strengthening your interests might be to have interdisciplinary learning. If you are able to apply an idea from one domain to another, it opens up a lot of perspectives to your research work.
You can recycle (and reuse) ideas from multiple sources.
Important Problems
Additionally, the creative process of solving problems or “asking the right question” also defines you as a researcher: problem-solver or problem-creator. In these two modes, one thing is common — identifying the important problems.
How to identify the "importance" of a problem?
It's a conundrum. However, there are a few ways to do so:
First, exploring different problems helps you to assess the importance of problems (such as assessing different research landscapes)
Secondly, working on lesser-known problems might also make them important. Since no one is working on them, you'd be able to get ahead in the game. (Just a tip of the iceberg though.)
In all these, one thing is sure: research work is intimidating and even an iota of self-doubt is unhealthy. Impatience and constant context-switching render your work as a “researcher” ineffective. Again, this relates to the very first point in proactivity too — Locus of Control. Too much negative feedback loop deprecates your research works.
For instance, you focus on minor problems that are already solved. You don't get chance to strengthen your core interests. You switch back-and-forth to many such problems. As a result, you are deprived of working on important problems.
Overall, this essay is a profound one. Probably, there should be a good balance of short-term and long-term interests (and goals). Life’s kinda messy and non-linear. So, being fixated to a particular mindset isn’t healthy. You should allow more flexiblity in your research processes.
This also made me revise some of the key ideas from Richard Hamming’s You and Your Research. (Also related: Reimagining the PhD)
Innovation vs Imitation
Listening | Shane Parrish | Derek Sivers | 1hr 40 min
Musician, speaker, writer and entrepreneur, Derek Sivers chats about creating and running CD Baby, reading, mental models, living a meaningful life and that biggest mistake he’s ever made.
This was such a good conversation to listen to. Derek's enthusiasm to talk about life, work, habits, and goals are contagious. His life's nuggets are also a great source of inspiration.
I loved his mental framework of Hell Yeah, or No!. If something doesn't make you feel “Hell Yeah”, it probably will not add any value to your life.
Fill life mostly with NO, but allow some YES in the spaces.
Another thought-provoking fragment from the conversation is treating ideas and execution as multipliers of each other. Even if you have a good idea, poor execution makes it less valuable. Or a mediocre idea with proper execution can create great values.
Ideas are the multipliers of execution. The most amazing idea with poor execution creates little values.
Overall the conversation swings between life, work, and goals which add a lot of interesting perspectives such as:
Don't be a bottleneck in your work. Segregate responsibilities and help others to understand your decision-making processes
Don't overpopulate your mind with future-tense thinking. (“I will be talking about this next month” :D)
It's better to have one great thing with your life than hundreds of less-valuable things.
Your questions on life can change your perspectives towards life. (See: Our emotional interpretation of an event directs the paths that follow.)
Consume information based on problems you are fascinated about, and thus the insights you get thereafter.
Imitate with innovation. Think deeply about simplicity. The world is unnecessarily ceremonial where people imitate without questioning anything.
Always prepare for the worst.
Shape your surrounding to best suit your priorities.
Learn with the finest precision. Be a connoisseur.
The Boredom Problem
Reading | Shiva Prabhakaran | 8 min
This read presents boredom, especially in workplaces, as a deteriorating factor in self-development. It can be external (like a boring environment, not getting any challenging work) to internal (like depression, anxieties, fuzzy goals).
It makes so much sense to me right now.
As someone with Shiny Object Syndrome (see: issue 16), I get bored too easily if not provided with challenging problems to work on. Perhaps, that also stems from a lot of anxieties and having a lot of dissatisfactions towards life in general, to not able to see the impacts I am having in the world. I don't know. It's not even burnout which results from stress and over-working on things. I jump from one problem to another to get a certain sense of kick. Mostly that kick arises with a lot of meta-thinking and meta-learning.
Lately, I have been feeling a lot of boredom, unable to spend my time productively. Life's kinda stuck in a rut, boring. Just yesterday, I spend 2 hours watching a tutorial on Ardour to improve my music recording skills. But I procrastinated on implementing it.
I have even stopped watching movies, reading comics, or playing guitar much. Existential dreads and loneliness are always in the loop.
I don’t know. When someone shares a link to a video/article, chances are I have already consumed that. I think that’s the perk of mindlessly browsing the internet, getting fascinated by almost everything. Eventually, boredom hits hard when I don’t find anything new. And I am again in the same “anxious loop” without motivation.
#Watching
Passage
6 min 40 sec
Passage, a short animation by the French filmmaker SimonFeat, limns twilight’s sensations with a delicate simplicity that renders the quotidian exquisite. The rhythms of terrestrial lifeboat cruising back to port, a cat settling down in the last rays of the sun, shadows creeping across a village, a cicada emerging from its shell– intertwine, pleasantly meandering through the end of a beautiful, unexceptional day. And then, as the sun dips below the horizon and dusk gives way tonight, the music shifts and we launch from this planet out into the great beyond.
This is one of the most beautiful animations I have ever watched. Enchanting.
This definitely puts perspectives on spans of life. It also nicely goes with this read too: Still life
This also reminded me of “The rhythms in everything”.
Do what you can’t
Casey Neistat | 4 min
Lately, I've been watching a lot of Casey's vlogs. I found this very inspiring.
Side note: I love all of his vlogs. His cinematography is on whole another level despite being "vlogs". Artistic.
Can you fit whole game in QR code?
MattKC | 20 min
This was actually fun to watch. Hilarious, yet informative. :D
The answer is YES. But it's a bit tricky; it needs a whole another level of coding skills, like knowing assembly language, compression, compiler optimizations, and such.
Here, he uses the largest version of the QR code (version 40) that can store around 2.88 KB of data. He goes onto fitting a snake game in it. It’s an interesting problem. Kinda cool.
#Fascinating
an ocean of books
It’s a website to explore the ocean of books to find your favorite author's islands and discover similar ones near them. Check out the David Foster Wallace island.
DeviantArt: give me your hands
A beautiful and trippy picture.
#Music
Haami - Yaad
Back to Yugal Gurung and the gang with this math-rock style. Totally underrated band. Also, I find their Reflection profoundly moving.
TEMPOREX - Care Full Album
Dreamy. The whole album is a gem for loners (like me). :)
re: Stacks
Bon Iver’s music is like pieces of soul scattered somewhere around. This particular song reminds me of a fragment of someone. :)
#Poetry
Singularity
I had first encountered this poem through this brainpickings post.
Powerful. A tribute to Stephen Hawking. The animation is equally beautiful; it does so much justice to the words and Marie's voice.
Also, this letter from Alex Flynn regarding the poem is beautiful.
#Ending-Thoughts
I have been pondering about purposeful-learning. One thought on top of my head these days is:
Does learning without any purpose exist?
What's even the point of "busyness" of learning without purposes?
One common ground I find with a friend of mine regarding this is “Collector’s Fallacy”. I guess life's a constant bet to stitch things together.
That's all for now. I really hope you found this post helpful. Please do share with someone that might find these things interesting; might motivate me to keep on sharing my thoughts.
This particular post took a bit longer than usual to write. Not sure if “time spent” correlates with “quality”. To be honest, I haven’t seen any increase in views in my newsletter except from a few close friends. That’s kinda demotivating. I really like to think that I shouldn’t care about metrics. Still, there isn’t other way to see if these letters are having any impact. For now, I just let it be.
Love,
Nish
PS: I loved this podcast: François Chollet: Measures of Intelligence | Lex Fridman Podcast 120. It also touches upon GPT-3. (I’ve been playing with GPT-3 for a week now, and have a lot of disappointments to share. Stay tuned.)
Oh BTW, I uploaded, yet another, shitty music - Drifting Somewhere. Give it a listen. Or drop by a few motivating feedbacks?
thanks for writing Nish and I really appreciate how you have been putting these out every week. This was another interesting topic that I never consciously pondered upon. Also, I realize that this is a big commitment of your time, and not every week is easy. Maybe sometime around you could share how you're doing this even in your busiest saddest of weeks (other than ok I just do it) and whatever it is that makes you read an article from 2004. Would be fun :)