Negative Feedback

Author: alchemyst

  • Novelty is easy, quality is hard

    I am often surprised when people claim that “AI can’t make anything truly new”. I think making new things is very easy

    Let’s try generating some novel text:

    fHabBiCcABnqjSAIysqcyJAnCTWCIk

    That’s very novel. I used Python to produce random letters:

    import random
import string

print(''.join(random.choice(string.ascii_letters) for _ in range(20)))

    Oh, but those aren’t even words, you say? OK, let’s try again:

    anthocyan Delhi iliococcygian methodologically opah reapparel pretimely humoristic Jingbai specks

    This was made by randomly selecting words from my system dictionary.

    import random
import pathlib

words = pathlib.Path('/usr/share/dict/words').read_text().splitlines()
print(' '.join(random.choice(words) for _ in range(10)))

    Very novel. I now hear someone say that this is not a well-formed English sentence.

    We now stand at a crossroads. We can enforce the structure of a sentence (something like “the <noun> <verb>ed the <noun>”) or we could work statistically off some corpus with simple Markov chains. These simple Markov processes can generate some surprisingly readable text (without using any rules about the english language). Here’s a snippet that was on top of the kingjamesprogramming tumblr site when I went there. This was generated by a Markov process trained on the King James bible and “The structure and interpretation of computer programs“.

    then shall they call upon me, but I will not cause any information to be accumulated on the stack.

    I’m pretty sure that’s a novel thought. But we’re starting to get to what people would perhaps agree is a sentence but start objecting to more on semantic grounds. Does this sentence mean anything? Is it true?

    If we go down the “The noun verbed the noun” route, we end up with a slightly different feeling. A feeling of “sameness”. I think often this is what people react to when they spend a bit of time conversing with ChatGPT. It’s not the novelty of the words or even ideas they’re responding to, it’s more the sameness of the vibe. Something that in people could be called their writing voice.

    This is a very deep concept, but can also be circumvented by leaning harder into randomness. In the GPT API, you can increase the parameter called “temperature” to get some more “out there” combinations.

    In the end we’re back to the same thing: It’s not the novelty that’s the problem. It’s the quality.

  • Don’t judge your open-loop predictions by closed-loop outcomes

    Many of us have driven vehicles with backup cameras that allow us to see where we would go if we kept the steering wheel at a particular angle. When the screen shows the lines colliding with an obstacle, we know we should behave differently: take this prediction into account and turn the wheel a different way.

    Backup camera view showing vehicle trajectory.

    These lines are an example of “open-loop” predictions. In other words, they show what will happen if you didn’t use the output of the model (no feedback). In contrast, what actually happens is a “closed-loop” outcome – you are using all your information to avoid the collision.

    I often hear people say that predictions were “wrong” because the bad outcome predicted didn’t happen. This is only valid if no corrective action was taken due to the prediction.

    We should rather evaluate predictions based on experiments where we fix the inputs completely. If a predictive model is doing its job right, and you take corrective action, the worst predictions will never occur, because you used the model to predict and sidestep them.

  • Transcribing birds on a wire

    I was on a walk the other day and saw these birds sitting on power lines and immediately spotted their resemblance to a musical staff.

    Birds sitting on power lines, resembling a musical staff

    I spent a bit of time massaging this into music. I assumed this was the G clef and that we were in C major. I made some “rules” like the birds sitting low on the wire wire “on the line” and the birds sitting high were in the space. That makes the first note a D and the second a G.

    I opened up Ableton Live and mapped out this sequence in the piano roll. Note I also had to make some assumptions about the length of notes. Things don’t line up perfectly, but I also used my ear to come up with a satisfying rhythm.

    Once I identified the main melody, I figured out chords that go well in the context and added a bass line. I’m trying to practice my MuseScore transcription skills, too, which resulted in this score:

    and, with some added drums, we end up with this little loop:

    I really enjoy this kind of constrained creativity prompt as it gives good inspiration and turns the creation process into little separate problems to solve.

  • Four years of sight reading practice

    I’ve been doing relatively frequent sight reading practice using an iPad app for four years. Here’s how I’ve automated some of the steps and what I’ve learned from the experience.

    Note: This post featured on Hacker News and from what I can tell spent a bit of time on the front page. There are quite a few insightful comments. If you are coming to this story after having read some of the comments, the most important thing you should know is that I do more than just this random note reading excercise to engage with music. Jump to the end to see it.

    Some back story

    I’ve been playing the guitar since an accoustic guitar was delivered to our scout hall by accident in the early 1990s. There was a piano at home, but for a variety of reasons, I didn’t play it. When my folks moved to a smaller house in 2021, the piano came to live in my study/music studio. I figured it would be a shame if I didn’t learn how to play it. I understood the basics of sheet music, but the decoding process was always very laborious, not helped by the fact that I had learned guitar mostly from tablature, which felt like a very different skill.

    Hardware and software

    I am constantly aware of fighting GAS by trying to use the things I already own. So I have been using the same M-Audio Axiom 49 key MIDI keyboard for years.

    I have a WIDI Master connected to its MIDI ports, so that I can connect to it via bluetooth on my iPad.

    This works flawlessly. The app I am using has gone through a few renames. It is as of 2025-05-11 it is called “NoteVision“. I can’t claim to have done a particularly exhaustive evalauation of music learning apps. This one just seemed to click for me due to the rapid feedback.

    NoteVision’s Grand Staff practice has options to set the range of notes (which I’ve set to match the limited range on my MIDI keyboard), a key signature selection, a “Chords” mode that switches between just single notes to showing common chords (basically triads, 1st and 2nd inversions), and a “Sharps and Flats” toggle that adds accidentals.

    The setup screen

    Once you click the play button, you get a full-screen staff and randomly selected notes, the bar moves as you play.

    The play screen
    The summary screen

    The routine

    I have written a little interface in Pythonista that I can start directly from my home screen. It selects a key for me to input into the iPad app, as well as tracking the other two options in the sight reading screen (whether there are full chords and accidentals in the generated notes). I’ve found it quicker to use the sliders than to input the numbers.

    Pythonista GUI for capturing results

    The app sends a REST request to my server which inserts the values into a MySQL database. This part of my music practice session takes about 10 minutes.

    Dashboard

    I have built a dashboard using D3 to visualise the data captured by the Pythonista app. I will release a separate post about the mechanics soon. I’m still learning D3 and figuring out what I can do, but I quite like the layout that I have converged on. The left hand panel shows the metrics over time. You can see some distinct phases in my approach on the chart below

    1. I started out focusing on the basics. C major. This is just the white notes on the piano, and I improved rapidly. There is a bump halfway through this phase where I turned on accidentals. At this time I was just keeping notes on an Excel spreadsheet.
    2. Next it was time to start with different signatures. I chose G major since I was also learning a piece in G major (a beginners piano version of Haleluja).
    3. It was clear that I needed to shake things up a after having hit a bit of a plateau on G major. I moved my spreadsheet over to Airtable and tried to choose a different key every day. I found myself favouring the sharps and struggling with the flats.
    4. I wrote the phone interface which helped me to randomise across all keys and took the approach that I just had to play what the app said. This was quite motivating as I could start to see the results coming in quickly. I also worked with my visualisation to identify keys that I was doing poorly in, the app slightly favours those keys in the randomisation. I update this periodically.
    5. As I got faster, I found that I could do more repeats in the same time. I am now doing 6 repeats instead of the 5 that I was doing at the start.

    On the right of the dashboard, I show the progress per key. It’s clear I’ve put in many more sessions in C and G. These days I rarely practice those keys, trying to get the others to catch up. I prioritise in my randomisation the keys that I have fewer sessions in and the ones I have lower accuracy in.

    The relevant part of the code is here. 

    if random.random() < 0.6:
	keys = [
	    'C',
	    'G',
	    'D',
	    'A',
	    'E',
	    'B',
	    'Db',
	    'Ab',
	    'Eb',
	    'Bb',
	    'F'
	]
else:
	# pareto front  
	keys = [
		"E",
		"Eb",
		"Db"
	]

    On 60 % of days I’ll play a completely random key. On the other 40 % I’ll play only the keys on the Pareto front. At this point I’m just eyeballing these, but I’d like to implement an automated Pareto front. I’m also not happy with the overlapping labels on the chart, but I’ll get around to that, later.

    What I’ve learned

    You don’t need to name notes to play them

    With the rapid feedback I get from the app, I quickly found that I was going directly from the annotated notes to the finger movements, without spending time decoding the note names on the way there. That means while I was learning to play the notes shown in any key signature, I was not learning to name the key signatures. This is because I am mostly counting the number of sharps and flats and translating that to the keyboard through a pattern I figured out early on. The sharps “activate” from left to right across the groups of black notes, starting with F♯, alternating between the two groups of black notes. This is easier to get into your fingers than any other memorisation technique.

    The order for flats is mechanically symmetrical – you just start from the right and move left, again from the “first” note in the group of three, which in that case is B♭. I am still not quite sure how other people are learning this, since most of the materials I’ve seen have focused on learning the actual names by rote, using mnemonics like “Father Charles Goes Down And Ends Battle”, but that still needs to be translated to fingers. I realised I needed a whole other drill for the naming, and settled on Anki.

    There is still progress, after four years

    I am surprised that I am still getting faster after all of this time. I guess another way of saying this is that my progress has been slow, but early on I was spending only around 10 minutes on these sessions and not doing much other practice most days. I have become significantly more confident in my sightreading ability when learning new songs. There is now no significant barrier between my eyes and my fingers when reading the music, although I still encounter issues outside the range I’ve practiced. It’s hard to tell how much of the additional progress I’ve made is down to other things I’m doing like playing songs (see later for my full routine).

    I would have liked an 88 key keyboard

    Due to the limitations of my MIDI keyboard, I only cover the middle range of the keyboard. This means I am considerably less fluent in the upper and lower range of the piano. I find myself having to work down octaves mentally, when I have to play low bass.

    Randomisation is excellent

    When I was selecting keys myself, it was clear that I mentally avoided the hard ones and was drawn to an easy C major day when I wasn’t feeling great. The randomisation means I don’t have that decision to make and makes sure I don’t zero in on wrong patterns. I suspect there is some difficulty on days where I shift from several days of sharp key signatures to flats, which I would still like to analyse.

    An extended practice routine

    For those of you thinking that I should do more than just random sight reading, of course you are right. I have a larger 30 minute plan which I am trying to get better at doing consistently, although I must admit I don’t get to it all most days.

    1. Sight reading practice (sets “key of the day”)
    2. Scales and arpeggios in the key of the day
    3. Theory and memory drills using Anki. I go through note names, key names and scales/modes
    4. Notation and transcription. This could be transcribing a random song from my Apple Music playlists or composing and annotating
    5. Ear training. I’m still researching a good iPhone app for this, but I try to work on playing songs by ear each day
    6. Repertoire practice. I have several pieces that I am trying to learn, so I choose one and work on a few bars every day.

  • Generative AI will help us mine the creative space faster

    With the important leaps being made in the world of AI generative art I’ve been returning in my mind to a short story by Spider Robinson called Melancholy Elephants. It had a significant impact on how I think about creativity. Most of what I say will be about music, two-dimensional visual art and the written word since these are the fields most affected by progress made in the last year or so. Not coincidentally, these are forms of art that have been most successfully digitised.

    Large is not infinite

    In the story, they are considering removing the limitations on the length of copyright. But automated copyright checking has become a reality and there are more and more people who want to produce art. Robinson did not anticipate the development of AI generated art, but the main point is still sound. The story turns on the tendency of humans to round large numbers up to infinity.

    How long can you send your sewage into an infinite ocean? Forever. How long can you send your smoke into an infinite atmosphere? Forever. We keep getting surprised when very large things turn out to be finite. 

    Robinson observes that our creative endeavour is limited by our perceptual machinery. In the story this is why infinite copyright combined with automated enforcement is a bad idea, because artists need to be allowed to rediscover forgotten art.

    How can art be finite? 

    Let’s start with the basics. The three types of art I mentioned before (words, sound and images) can all be digitised. Literally, that means they are reduced to numbers. Every file on your computer is a number – a sequence of bits. For finite-length files, there is a finite number of combinations of bits. But this space – the space of all possible files smaller than 5 MB, say, is really quite large. Like atoms in the universe large. Unfortunately that is not the size that is important.

    Consider that a picture like the Mona Lisa can be represented by pixels on a screen. At what resolution can we still recognise the Mona Lisa? What amount of noise can be added, what colour changes, before you declare the new image to be novel instead of a derivative work? Similarly, you wouldn’t have made a new song if you simply took the first minute of Stairway to Heaven, or appended two copies of it back-to-back. These are “new numbers” but they are also “the same” as other existing numbers.

    There is a great deal of confusion around attribution, recognition, novelty, and the protections provided by the law. Copyright, considered by itself, is specifically a restriction on copying. However, what constitutes a copy of a story, image or piece of music is becoming more broadly defined all the time.

    These questions about what kinds of mutations constitute copying are very interesting. In the literary space, the issue of retelling a story is well understood. However, in the visual arts, the issue is different. People still value the uniqueness of an original painting. The provenance is important. You couldn’t demand the same price for a replica, no matter how good, if you clearly stated that it was a replica. However, in terms of appreciation of art or images, there is a much bigger market for electronically reproduced images than there is for paintings.

    In the musical space, the process of musical notation or the representation of a musical piece in a computer file allows for the reproduction of a performance to a high degree of accuracy. Many of us consume music mostly through computer means or recordings, and the connection to live performance is less emphasized.

    This means that the copyright granted on these forms of art is not only specific to one particular representation, but also a large cloud of numbers that are related to that representation through electronic operations.

    So the space is made smaller by the fact that theoretically different points are practically considered similar.

    Additionally, most of the space represent combinations of words, sounds or images which are not pleasing to humans. This is much like the way that most of the earth is filled with “useless” rock, while the valuable gold or diamonds occur in reasonable concentration only in a small number of places and need to be mined. A good way to get a feeling for how sparsely the good bits are populated is to imagine randomly generating a page of text. If you just chose letters at random, you would be exceedingly unlikely to create even one sentence that made sense, let alone have some literary value.

    Technology improves our exploratory speed

    My view is that as new technologies for producing images, music, or words emerge, they allow us to more rapidly explore the creative space, focusing on the good bits. There is nothing like creativity in the sense that that word implies, because we do not create these pieces of art, we just discover these pleasing locations in the space. Our intuition about these creative spaces is often informed by a first arrival principle. We stake our claim to a particular point or a particular number and that gives us ownership of the number and its immediate neighbourhood. It used to be that to discover a place, it had be playable on an instrument made for humans. With sequencers, synthesisers and other modern audio technology this is no longer a real constraint. 

    For example, when electronic sound synthesis was developed, we had an explosion of new music. Changes in methods for reproduction also have an impact on music. Music used to be built to be radio-friendly, resistant to being filtered by radio, and still sound good over poor quality speakers. As the reproduction methods improve, that allows some previously unexplored space to be mined. 

    Modern generative AI techniques have exploded many of the constraints that used to hold on these forms of art. Anybody can supply a prompt to ChatGPT and get a decent starting point for an essay. Midjourney creates epic artistic pictures of characters or landscapes, and there are several recent advances in generating music. They already allow us to generate convincing lounge music, that can be indistinguishable from live performance, and it challenges our understanding of what constitutes originality, creativity, and ownership in art.

    The development of generative AI techniques in the field of art is rapidly advancing, and this progress is causing a shift in the way we think about art and the skills required to create it. 

    An analogy can be drawn to the impact that digital computers have had on the field of arithmetic, where human computation has been replaced by machines. Similarly, as generative AI techniques become more advanced, certain skills that were previously required for artists may no longer be in demand. This trend may lead to a decrease in the number of jobs in the field of music production, and a shift towards filtering and trade, where pre-existing art is consumed, rather than investment in “new” music. 

    However, there may also be an initial spike in novelty produced by these algorithms, which can be exploited by artists. Furthermore, the proliferation of AI generated art may change the way we think about art and its creation. As the technology continues to advance, we may begin to see vast sections of the creative space being mined by AI. Additionally, it is important to note that most of our experiences with art are now transmitted through digital mediums, and this shift in technology will likely continue to shape the way we consume and interact with art in the future.

    What about the emotional component?

    A common counter-argument I hear when I talk to people about this topic is that they appreciate the emotional connection to an artist. The suffering of van Gogh or Kobain speaks through their works. I think the romantic notion that you can feel the pain of the artist through their work is mumbo-jumbo.

    There’s nothing in the art but the art. To the extent that you are particularly impressed by an artist’s pain, you would have had to have learned that through another channel. I think we are very close to AI producing convincing pieces of art that allow people to have very satisfying emotional interactions with them. 

    A moment’s introspection will also allow you to understand that the transmission of emotion model must be  wrong. Just consider how many touching or moving moments you have experience from advertising I’m sure most people would agree that the people on the other side are not necessarily having a bona fide emotional connection with the audience any more than a good well-trained computer model without doing instrumental art.

    In other words, art that is trying to make you feel something or art that is trying to make you buy something is not that different.

    Art is a controversial term in this discussion because people bring in a whole lot of extra baggage that they connect to that word. I think they over-emphasise the importance of the artist’s intent.

    But, in fact, I think we should focus on the artefact, in other words, focus on the experience of being subjected to this particular set of numbers and ask yourself how much it means for an artist to be a real human.

    Can you tell? And if you can’t, does it matter?