# Jupyter+ipwidgets GUI for Nilpotents to Permutation Conjugacy Class Correspondence

I was looking for a family of symplectic nilpotent matrices with prescribed associated permutation conjugacy classes. So I made a GUI in Jupyter+ipywidgets to go over matrices, and it actually worked.

Here is the code the GUI:

In [1]:
from collections import Counter
import numpy as np
import ipywidgets as widgets

In [2]:
def get_k(A):
"""Get nullity sequence of A"""
n = len(A)
k = []
for i in range(2 * n + 1):
nullity = 2*n - np.linalg.matrix_rank(np.linalg.matrix_power(A, i))
k.append(nullity)
if nullity == 2 * n:
break
else:
return None
return np.array(list(filter(bool, np.diff(k))))

def k_to_perm(k):
"""Convert nullity sequence to permutation conjugacy class by taking dual and fancy python"""
if k is None:
return None
perm = Counter()
for i in range(1, max(k) + 1):
perm += Counter({np.sum(k >= i)})
return perm

def pretty_perm(p):
"""Return nice two-rowed representation of permutaion conjugacy class"""
if p is None:
return 'not nilpotent'
top = ''
bottom = ''
for i in sorted(p.keys()):
b = str(i)
t = str(p[i])
bottom += b + '  '
top += (' ' * len(b)) + t + ' '
return '%s\n%s' % (top, bottom)

def favorite_symplectic(n):
w = np.zeros((n, n), np.int64)
for i in range(n//2):
w[i, n - i - 1] = 1
for i in range(n//2, n):
w[i,  n - i - 1] = -1
return w

def favorite_orthogonal(n):
w = np.zeros((n, n), np.int64)
for i in range(n):
w[i, n - i - 1] = 1
return w

def show_grid(n, preserve=None):
"""Show nxn grid of buttons and a textarea for permutation conjugacy class"""
if preserve is not None:
if preserve == 'symplectic':
preserve = favorite_symplectic(n)
elif preserve == 'orthogonal':
preserve = favorite_orthogonal(n)
w_inv = np.linalg.inv(preserve)

A = np.zeros((n, n), np.int64)
bs = []
txt = widgets.Textarea()

for i in range(n):
row = []
for j in range(n):
w = widgets.Button(description='0', layout=widgets.Layout(width='5px'))
w.style.button_color = 'red'
def foo(b, i=i, j=j):
if b.description == '0':
v, c = 1, 'lightgreen'
else:
v, c = 0, 'red'

A[i, j] = v
b.description = str(v)
b.style.button_color = c

if preserve is not None:
M = np.zeros((n, n), np.int64)
M[i, j] = 1
M = -np.dot(np.dot(w_inv, M.T), preserve)
(i2,), (j2,) = np.where(M)
if (i, j) != (i2, j2):
A[i2, j2] = int(M[i2, j2] * v)
bs[i2][j2].description = str(A[i2, j2])
bs[i2][j2].style.button_color = c

txt.value = pretty_perm(k_to_perm(get_k(A)))
w.on_click(foo)
row.append(w)
bs.append(row)

display(widgets.VBox([widgets.HBox(row) for row in bs] + [txt]))

In [3]:
show_grid(4, 'orthogonal')


# Adding GUI to Sequential Python Scripts Using Generators

* This post was written in jupyter and then pasted into wordpress *

In this post I will explain a method I came up with at work to add a graphical user interface (GUI) to a sequential python script – using generators.

Most tutorials on python generators focus on the advantage for writing iterators. For example the first line in https://wiki.python.org/moin/Generators is:

"Generators functions allow you to declare a function that behaves like an iterator, i.e. it can be used in a for loop."



That’s true and great. But in this post I will not use generators for iteration (at least not explicitly), but rather for their special ability to halt a function and later resume it.

At work we do a lot of exepriments with sound and temperature, and we write sequential scripts for them that we run in a terminal (or idlespork, my favorite IDLE fork). When we started working with volunteers, we wanted them to see some graphs on the screen, but also we didn’t want them to see an ugly terminal. We needed to add a GUI. The point I want to make is this:

Experiments scripts are sequential in their nature:

1. Enter details
2. Turn on oven and press enter
3. Turn off oven and press enter
4. Enter some data
5. Goodbye!



Usually, on the other hand, GUI is not sequential. It is object-oriented and event-driven. And that’s great of course. I love writing GUI in the usual way (I’m kidding, no one likes to write GUI). So how do we take working sequetial python scripts that need user input and add a GUI to it, with as little work as possible?

## Step 0 – Have some code

The running example of this post will be a small script that lets you compute the number of beats per minute (BPM) of a song by manually pressing enter on each beat for, say, 30 seconds. Here it is:

In [1]:
import time

def bpm():
print("[  0] press enter on every beat ", end='')
input()

s = time.time()
beats = 1

print("[  0] press enter on every beat ", end='')

while input() == '':
print("[%3d] press enter on every beat " % (beats*60/(time.time() - s)), end='')
beats += 1


The code is simple and sequential:

1. wait for first enter press,
2. wait for more enter presses,
3. on every enter press calculate the new BPM approximation.

I put on Benny Goodman and Charlie Christian’s A Smo-O-Oth One and ran it:

In [3]:
bpm()

[  0] press enter on every beat
[  0] press enter on every beat
[118] press enter on every beat
[126] press enter on every beat
[127] press enter on every beat
[129] press enter on every beat
[129] press enter on every beat
[130] press enter on every beat
[128] press enter on every beat exit


Writing this post was the first time I used input() in jupyter, and I was happily surprised to find it works.

## Step 1 – Refactor screen printing

Displaying stuff is different when using a terminal than when using a GUI. By refactoring out the printing parts of the function, we get ready to replace them with a GUI statement.

In [4]:
def display_terminal(value):
print("[%3d] press enter on every key " % value, end='')

def bpm(displayfunc=display_terminal):
displayfunc(0)
input()

s = time.time()
beats = 1

displayfunc(0)

while input() == '':
displayfunc(beats * 60 / (time.time() - s))
beats += 1


## Step 2 – Yieldify / Refactor out user input

Here we start to use generators. Instead of telling the script to wait for user input inside our function, we will have something external to get input, send it to our function, and then our function will yield back control to the external function.

This is how this looks, with comments indicating the two changed lines:

In [5]:
def bpm_gen(displayfunc=display_terminal):
displayfunc(0)
# input()
yield

s = time.time()
beats = 1

displayfunc(0)

# while input() == '':
while (yield) == '':
displayfunc(beats * 60 / (time.time() - s))
beats += 1

def bpm_loop():
gen = bpm_gen()
last_input = None
while True:
try:
gen.send(last_input)
last_input = input()
except StopIteration:
break


The function bpm_loop creates a bpm generator and then relies on the send mechanism to feed the generator with user input. This is where we’re using a generator in a slightly different way than normal – instead of iterating, we keep sending values.

The major point here is this:

We have converted waiting for user input in our function to waiting for our function in a loop that gets user input

This is possible due to the ability of generators to halt and later resume.

## Step 3 – Add a GUI

We are ready to add actual GUI components. All we need to do is replace the bpm_loop function above with something else that calls gen.send.

This is how this looks with Tkinter:

In [6]:
import tkinter

def bpm_gui():
# Create a window
top = tkinter.Tk()
top.minsize(width=400, height=50)
top.maxsize(width=400, height=50)

# Create a button
beat = tkinter.Button(top, text="[  0] click on every beat", font="courier")
beat.place(x=0, y=0)

# New display function that sets the button label
def display_label(value):
beat.config(text="[%3d] click on every beat" % value, font="courier")

# Create a bpm generator
gen = bpm_gen(displayfunc=display_label)
# Start the generator, i.e. get to first yield
next(gen)

# Configure the button to call gen.send on each click
beat.config(command=lambda: gen.send(''))

top.mainloop()


bpm_gui creates a Tkinter window, adds a button, creates a bpm generator that displays BPM approxiamtions in the button’s label, and finally configures the button to call the generator’s .send.

That’s it! We have added a GUI to a sequential script with as little change to the original function as possible. You can imagine how these three steps will work for basically any sequential terminal script. For textual user input (and not just pressing enter), we factor out the input statement to a function that reads GUI text components, and then it can hide them and reorganize the main window. Even if we do this, we still keep our core logic sequential – we don’t have to encapsulate the core in a class, initialize members, and change the way we think of the code.

## Step 4 – Run in a seperate thread

For the running example there’s no problem of speed, but at work we were playing music through headphones for a few seconds after each user input. This caused the GUI main loop to be stuck and it would show signs of dying (on Ubuntu the application window goes dark, on Windows you get the alarming “Not responding” in the title).

To fix this we must make sure the core code, which is now in a generator, be run in a seperate thread. I implemented this by sharing an Event object between the two threads: the logic thread would wait on the event, and the GUI thread would set it when needed. There are more ways to do this and it really depends on the type of the user input you have.

In any case the code to communicate events can sit in the external functions – the core logic doesn’t need to be changed again. It remains sequential and pretty close to the original code you had.

## Step 5 – Adding more stuff / Going overboard

Now that the original code works with a GUI, I can actually add something that I’ve wanted for a while: to see the distributon of BPM approximations with every beat. I.e. draw a matplotlib histogram figure on a canvas.

In [7]:
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg
from matplotlib.figure import Figure
import matplotlib.pyplot as plt

def bpm_hist_gui():
top = tkinter.Tk()

# Create button
beat = tkinter.Button(top, text="[  0] click on every beat", font="courier")
beat.grid(row=1, column=1, sticky=(tkinter.W, tkinter.E))

# Create a canvas
fig = Figure(figsize=(5,4), dpi=100)
canvas = FigureCanvasTkAgg(fig, master=top)
canvas.get_tk_widget().grid(row=2, column=1, sticky=(tkinter.W, tkinter.E))

# Variable to keep bpms
bpms = []

# New display function that sets the button label
def display_label(value):
beat.config(text="[%3d] click on every beat" % value, font="courier")

if value > 0:
bpms.append(value)

if len(bpms) > 10:
# Draw new histogram
fig.clear()
p = fig.gca()
# I actually want to see the BPM as calculated by every two consecutive beats/clicks.
diffs = [1/((i + 1) / b2 - i / b1) for i, (b1, b2) in enumerate(zip(bpms[:-1], bpms[1:]))]
p.hist(diffs, 20)
p.set_xlabel('BPMs', fontsize = 15)
p.set_ylabel('Frequency', fontsize = 15)
canvas.show()

# Create the generator
gen = bpm_gen(displayfunc=display_label)
# Start the generator, i.e. get to first yield
next(gen)

# Configure the button to call gen.send on each click
beat.config(command=lambda: gen.send(''))

top.mainloop()


# Christmas to Christmas With Audible

For Christmas 2014, Mihaela gave me the best Christmas present I had gotten so far (and the first I remember, which makes some sense, growing up as a secular Jew) – A year’s subscription to Audible, a book per month. Since I walk around a lot, drive, and often go on day trips to catch a flight, audio is a perfect medium for me. So here are the books and lecture series I got with my subscription, ordered chronologically, and after them books and lecture series I got without my subscription:

1. This Explains Everything: Deep, Beautiful, and Elegant Theories of How the World Works, Edited by John Brockman (12 hrs and 4 mins) – 150 little snippets of ideas from some of the most prominent thinkers of today. Honestly, it’s pretty hard listening to this. It kind of failed in its purpose (for me). I couldn’t really listen to this while walking or driving since it took so much concentration for one snippet, and then the next one was on something completely different. I regard this as convincing proof that the unconscioussness exists. I enjoyed learning about evolutionary stable strategies: why in all species there’s about a 1:1 ratio of males to females. Oh, and there are almost as many people talking about natural selection as there are those who say that they won’t talk about natural selection, leaving it to someone else.
2. Einstein’s Relativity and the Quantum Revolution Modern Physics for Non Scientists 2nd Edition, The Great Courses, Richard Wolfson (12 hrs and 20 mins) – If you’re not familiar with the Great Courses company, I recommend you check them out. I’m a big fan. In this course I learnt about the experiments and thought experiments that started relativity and quantum theory. Pretty cool stuff. Like how time dilation was confirmed with two clocks in a tower. One small problem with the presentation is that the professor shouts a lot when he gets excited. But he’s ok.
3. How Music Works, John Powell (8 hrs and 6 mins) – I thought there would be some more “how”. But yeah, I learnt some stuff. The nicest thing was to hear a banjo playing with the attack of each note removed. I heard of the phenomenon before, but didn’t experience it. Don’t know what I’m talking about? Then click… Nope. I couldn’t find a video showing this. Essentially, taking off the start of every note from a recording completely changes what instrument I think I’m listening to.
4. How We Learn, The Great Courses, Monisha Pasupathi (11 hrs and 42 mins) – This is mostly a long list of ideas intertwined with experiments. I really like this kind of combination, with Plato pointing up and Aristotle pushing down, and I get to understand it all. The studies on how language affects thinking were enlightening, for example the one comparing how English and Korean speakers attend in language and in thought to how objects can fit in each other (McDonough, Choi, Mandler 2000) – language affects thought, but maybe it can be easily fixed. Small problem though, it seems that the audio was edited to have many silent bits, like two seconds of complete silence. Maybe she drank some water or coughed or I don’t know what. But this is disturbing. It feels weird to suddenly hear completely nothing, like you’ve went into a vacuum tunnel. But at x1.5 speed it’s not so bad…
5. The Darwinian Revolution, The Great Courses, Frederik Gregory (12 hrs and 7 mins) – I had no idea. Well, I had some idea. I had heard the words evolution and natural selection before, but that’s about it. From the start I was surprised to find out how many people before Darwin thought about evolution, only missing out the final, crucial piece of natural selection. It’s also pretty cool that these days we know of epigenetics, which is pretty Lamarckian in my mind. Yey for Lamarck!
6. Memory and the Human Life Span, The Great Courses, Steve Joordens (12 hrs and 2 mins) – I didn’t finish this one. The guy’s a bit boring. There isn’t enough a narrative of history, i.e. what experiments and studies were done that told us something.
7. Biology: The Science of Life, The Great Courses, Stephen Nowicki ( 36 hrs and 36 mins) – It’s pretty long… I’ve left it and returned to it over seven months, and I’m about half way through. The course is usually interesting, but I think there are some hurdle-like lectures. I’m happy to say that I now really understand who Dolly the sheep was and how she came into this world. DNA is obviously amazing, but I’m also very impressed with how it was discovered, essentially just from a few images. This makes me think of Plato’s cave, and why I think he got it wrong: even if they showed us just shadows on a wall, sooner or later we’d be able to say what and how these shadows reflect, even if we can’t see the source directly.
8. One More Thing, B. J. Novak (6 hrs and 20 mins) – Finally, fiction! At some point I realised that Mihaela wasn’t enjoying the lectures series as much as I was when we drove places. So I looked for fiction, short stories in particular. This book has a good balance of surrealism and cursing. But unlike lecture series on science, I wouldn’t want to spoil any of the stories for you. Suffices to say that the audio snippet you can preview on the Audible website does not capture how much humour there is in the book. Which is a lot.
9. 36 Arguments for the Existence of God, Rebecca Newberger Goldstein (15 hrs and 34 mins) – A work of fiction about some philosophical ideas. It also touches on what is community, our sense of it, and how annoying accademia can be. Close to the book’s ending there is a debate between two characters, the main and a new one. It’s kind of weird. I’m not entirely sure what I think about this book.
10. A Primate’s Memoir: A Neuroscientist’s Unconventional Life Among the Baboons, Robert Sapolsky (14 hrs and 35 mins) – I’ve already talked about this book here.
11. The History of Jazz, 2nd Edition, Ted Gioia (21 hrs and 59 mins) – I’m still at the beginning. It’s just a little bit boring. But I’ve heard it gets better and all in all is very interesting stuff.

During the year I also listened to a few other books and lecture series, not through my Audible account:

1. The Man Who Mistook his Wife for a Hat: and Other Clinical Tales, Dr. Oliver Sacks (9 hrs and 36 mins) – Each tale is one to remember, and they all made me think. Mihaela finds Sacks’ going on and on a bit tiring sometimes, and I partially agree with her. But c’mon, this book is awesome. It has introduced me to the outskirts of mental health in a most human way. I particularly like the story about the ex-soldier with Korsakoff syndrome, who can’t remember anything new, and yet knows the layout of the garden outside the facility.
2. Beethoven’s Piano Sonatas, The Great Courses, Robert Greenberg (18 hrs and 23 mins) – The professor spends enough time to build my understanding of each of them. I’m delighted to know that no. 24 is one of Beethoven’s favorite, mostly for its simplicity, contrasting it with the rest. I would have enjoyed lengthier discussions on no. 31, which is one of my favorites (if I had to name two, I would say nos. 31 and 32).
3. No Excuses: Existentialism and the Meaning of Life, The Great Courses, Robert C. Solomon (12 hrs and 7 mins) – Yes! I can now understand much more Existential Comics! I really like Camus’ reaction to the absurd. But I am conflicted with regards to Sartre’s idea of responsibility. I’m just not sure what that means. Videos like this one, with Robert Sapolsky and Alan Alda, only make it worse (though it is fun to think about all of this).
4. Stress and Your Body, The Great Courses, Robert Sapolsky (12 hrs and 19 mins) – Loved it. I’ve already talked about it here.
5. Plato, Socrates, and the Dialogues, The Great Courses, Michael Sugrue (12 hrs and 2 mins) – Yes! Now I understand even more Existential Comics. But really, I understand quite a lot more. A great advantage I got from this course is knowing to attribute many ideas I had heard of before to Plato’s Socrates. This has helped organise my thoughts on many things, and also with the ability to now search and learn more. I like Plato’s idea of the doctor – one who knows what is sickness, but also, crucially, what is health. Thinking of modern psychology, for example, this leads to requiring the knowledge of positive psychology and not just pathology. In this context Plato’s idea is very concrete.
6. Particle Physics for Non-Physicists: A Tour of the Microcosmos, The Great Courses, Steven Pollock (12 hrs and 10 mins) – After listening to the relativity and quantum theory lectures I knew that I would have to find more stuff to listen to in order to satisfy my new physics lust. Nathan showed me some lectures on YouTube on particle physics that seemed intriguing, so I knew what to look for. The course is pretty cool, again with a good combination of ideas and historical experiments. I was impressed by the neutrino experiments, putting large bodies of chlorine inside a mine and literally getting an image of the centre of the sun. The professor is enthusiastic and stuff, but he got on my nerves a few times. He called Aristotle an armchair philosopher. He said Democrats was wrong, since atoms can be divided (though later in the course he implicitly corrects this). He keeps calling Emmy Neother by her full name, mispronouncing it, even though he calls all males “Mr.”. And why is he calling them “Mr.”?
7. Bossypants, Tina Fey (5 hrs and 30 mins) – It’s cute. Funny at times. The only story that made me lough out loud is the one about her daughter buying a book about a working mother. The punchline is fantastic.
8. The Martian, Andy Weir (10 hrs and 53 mins) – I don’t know why I listened to this. I think it was an attrition game against myself.

All together, without including repeat listens, this adds up to 194 hours and 13 minutes of listening. On average this means 31 minutes and 56 seconds every day. Huh. In the introductions of The Great Courses the guy says something like “imagine what you can accomplish with just half an hour every day”. I guess I don’t need to imagine anymore.

Here’s what I’m listening to now and what I plan to listen to soon:

1. The Will to Power: The Philosophy of Friedrich Nietzsche, The Great Courses, Kathleen M. Higgins, Robert C. Solomon
2. The Better Angels of Our Nature: Why Violence Has Declined, Steven Pinker
3. Behavioral Economics: When Psychology and Economics Collide, The Great Courses, Scott Huettel
4. The History of the United States, 2nd Edition, The Great Courses, Allen C. Guelzo, Gary W. Gallagher, Patrick N. Allitt
5. The Brothers Karamazov, Fyodor Dostoyevsky
6. Awakenings, Oliver Sacks
7. The Language Instinct: How the Mind Creates Language, Steven Pinker

Have you listened to anything good recently? Drop a comment!

# DJing From My Phone

Last Friday was the second time I DJed at a lindy hop party from my phone. The first time was in January, at a bar in Copenhagen during a local weekly swing night, after a wonderful weekend of teaching taster classes with Lee at the fantastic festival LHTA. That time I had used my special iOS pyhton program (written in Pythonista) and played from my small-screened iPhone 4S. This time, in Ljubljana,  I used my huge-screened Note 4, and a program I whipped up in the two days leading to the party. Did I test the program thoroughly beforehand? No. Was my set ok? Umm, yeah, sure.

Here are the abilities I need for a (good) set:

1. External sound card
2. Preview songs
3. Queue songs
4. Lock buttons on playing device
5. Search in media library by title, artist, bpm and comments.

This project of playing from my Note 4 that runs Android started when I realised that I can connect an external usb sound card to it using something called an OTG cable, as in On-The-Go. Brilliant! This solves point no. 1.

A few months ago I spent quite some time in trying to write a native iOS app that will play two songs in the two channels of headphones. There already exists such an app in the app store, but it’s expensive, doesn’t have a lock buttons feature, doesn’t search in comments, and it has turn tables. Enough with the turn tables already! I’m not that kind of DJ… Anyway, I didn’t figure it out before I gave up, and I was sad. But if I play music form my phone – I can just preview on my iPad, using my old python program. Great. This solves point no. 2.

I made the program with two tabs: one to show the media library and one the current playlist. Clicking a song in the media library tab queues a song in the playlist tab, and clicking a song in the playlist tab starts to play it. Unless the lock is on. Which is crucial. Personally, I am always worried about accidental clicks. It doesn’t matter if I’m using my phone, my iPad, or even my MacBook. Having a program that just ignores all clicks, when a lock is on, reduces my anxiety by a big factor. And I know a few other DJs who have confessed to the same stress. But when the lock is on I can still queue songs and change their position in the playlist. So points 3 and 4 are good too.

Of course I’ll need to search for songs. If I’ve already previewed a song on another device and I just want to add it now then I can search by title. But I really like seeing my comments… So I made them show up on the screen. And if they’re there, I should also be able to search the comments tag as well. What if I’m riding shotgun and have to provide the music? I’m not going to preview songs. Being able to search for a low energy groovy song would be nice.

Here’s a screenshot from the end of my set:

Though everything was solid, the set was stressful for me. I wasn’t sure the app would hold up with no crashes, and if the battery would survive supplying power to a usb card and having the screen on almost constantly. I’m happy to say that the app didn’t crash, and the battery only went down by 20%! That’s it! Granted, I turned off everything else and went into airplane mode (can you imagine getting a call in the middle of a set?). The only snafu was during the first song: I forgot that I made a lock-all-buttons button, and my headphones cable accidentally touched the phone’s screen – that’s it, it just kinda grazed the screen for a split second. And the song changed. Somehow, it was kind of on beat, and with no pause. So it was ok, and then I clicked the lock button.

One more thing about my set: following this post, I decided to use a normal distribution for my bpms. I had python opened up on my MacBook (which I brought fearing that my phone would explode and I would need another device) with numpy imported, and I got my bpms from its random module. For the first half hour, which was between band sets, I used a mean of 170 and sigma 16.66. Then, after the band went off for the last time around 1:20 AM, I reduced the mean to 150. Finally, after an hour and until the end of my set I set the mean to 140. Only two times during the set did I decide to overrule the random procedure. Since I had say in the energy and subgenre of songs, no bpm was supposed to be problem. Except for those two times, that two songs above 190 were just played, and I wanted a slower song to put on next. Overall, I think the room was pretty much swinging and bouncing until the end, at which point a jam of three musicians started on stage. The highest bpm was 202 and the lowest was 124.

Being able to go have a set without carrying a heavy computer is awesome. Once I add some features, like tagging and saving playlists and maybe even previewing, I will upload the app to the Google play store. But who knows when that will be…

# Quadratic Twists for Representations of Finite Symplectic Groups

I started mentioning descent in this post. During the course of writing my masters results in article form, I found a cute descent-like result that can be proven with similar ideas.

Theorem. Let $\sigma$ be a representation of $Sp_{2m}(\mathbb{F}_q)$ in general position. Then ${\textup{Ind}_{P_{1,2m,1}}^{Sp_{2m+2}(\mathbb{F}_q)}(1_{\mathbb{F}_q^\times}\times\sigma)}$ is reducible and has a unique semisimple component, call it $\pi(\sigma)$. Then

$\tilde{\sigma}:=(\textup{Res}_{Sp_{2m}\ltimes H_{2m}}(\pi(\sigma))\otimes\omega_\psi)^{H_{2m}}$

is the “quadratic twist” of $\sigma$, i.e. the characters in its Deligne-Lusztig parameter are the same, but multiplied by the quadratic character. A representation is said to be in general position if its character is plus or minus a Deligne-Lusztig character in general position.

Here $\psi$ is an additive character of $\mathbb{F}_q$, $\omega_\psi$ is the Weil representation of $Sp_{2m}\ltimes H_{2m}$, and $P_{1,2m,1}$ is a rational parabolic subgroup with Levi subgroup isomorphic to $\mathbb{F}_q^\times\times Sp_{2m}(\mathbb{F}_q)$.

The strategy of proof is as follows:

1. Express $\pi(\sigma)$ as a combination of Deligne-Lusztig characters.
2. Compute the degree of $\pi(\sigma)$.
3. Deduce a bound on the degree of $\tilde{\sigma}$
4. Compute the character value of $latex \tilde{\sigma}$ at all non-unipotent elements.
5. Deduce the dimension of $\tilde\sigma$.
6. Deduce character values at unipotent elements.

The first four points are pretty straightforward computations with Deligne-Lusztig characters. The fifth point is an application of the following crucial lemma:

Lemma 1. Let $\textup{\textbf{G}}$ be a connected reductive algebraic group over $\mathbb{F}_q$ with Frobenius morphism $F$. Then, as $n\rightarrow\infty$, for any pair of representations $\rho_1,\rho_2$ of $\textup{\textbf{G}}^{F^n}$, such that ${\textbf{dim}\ \rho_1\ne \emph{dim}\ \rho_2}$ and ${\chi_{\rho_1}-\chi_{\rho_2}}$ is zero on all non-unipotent elements, we have:

$\emph{dim}\ \rho_1\oplus\rho_2 \gg \sqrt{|\textup{\textbf{G}}^{F^n}|(q^n)^{r(\textup{\textbf{G}})}}$

where the implied constant depends only on $\textup{\textbf{G}}$.

In our case, $\rho_1$ is $\tilde\sigma$, and $\rho_2$ is the quadratic twist of $\sigma$. Assume that $q$ is large, and that ${\emph{dim}\ \rho_1\ne \emph{dim}\ \rho_2}$. Then the hypotheses of the lemma are satisfied. But then the conclusion of the lemma contradicts the bound on the dimension of $\tilde\sigma$. So, for all large enough $q$, we must have ${\emph{dim}\ \rho_1= \emph{dim}\ \rho_2}$. But the dimension of $\tilde\sigma$, as well as the dimension of the quadratic twist of $\sigma$, are polynomials in $q$ ($\sigma$ going over a family of Deligne-Lusztig characters in general position for the same torus). Hence, for all $q$ we have equality of dimensions.

We get the desired result by applying the following lemma:

Lemma 2. Let $\rho_1,\rho_2$ be two representations of a finite group $G$ with equal dimensions, such that their characters agree on a subset $H\subset G$, and that $\rho_2$ is irreducible. Assume there exists a $z\in Z(G)\cap H$, such that $z\cdot(G\backslash H)\subset H$. Then $\chi_{\rho_1}=\chi_{\rho_2}$ on all of $G$.

Our result says that we can twist representations of $Sp_{2m}(\mathbb{F}_q)$ with an explicit representation theoretic construction. I am not aware of such a result over local and global fields. All proofs that I have seen for twisting general automorphic forms use converse theorems. I will note that Shimura gave a geometric meaning to twisting modular forms by a general Dirichlet character, but being geometric in nature precludes it from working for Maass forms.

——————

Proof of Lemma 2: Since $\rho_2$ is irreducible, and $z\in Z(G)$, $z$ acts as a scalar on $\rho_2$, say $\lambda_z$. So

$\chi_{\rho_1}(z) = \chi_{\rho_2}(z) = \lambda_z\chi_{\rho_2}(1)=\lambda_z\chi_{\rho_1}(1)$

which implies that $z$ also acts as $\lambda_z$ on $\rho_1$.

Let $g\in G\backslash H$. Then we have

$\chi_{\rho_1}(g)=\chi_{\rho_1}(z^{-1}zg)=\lambda_z^{-1}\chi_{\rho_1}(zg)=\lambda_z^{-1}\chi_{\rho_2}(zg)=\chi_{\rho_2}(g).$
$\blacksquare$

Proof of Lemma 1: Let $n$ be a positive integer, and $\rho_1$,$\rho_2$ be as in the statement of the lemma. The assumption ${\textup{dim} \rho_1\ne \textup{dim} \rho_2}$ implies that ${(\chi_{\rho_1}-\chi_{\rho_2},\textup{reg}_{\textup{\textbf{G}}})\ne 0}$. Using Corollary 7.7 in [DL], there exists a pair $({\textup{\textbf{T}}},\theta)$, $\theta\in\widehat{{\textup{\textbf{T}}}^{F^n}}$, such that

$(\chi_{\rho_1}-\chi_{\rho_2}, R_{{\textup{\textbf{T}}}}^{\textup{\textbf{G}}}(\theta)) \ne 0.$

Since ${\chi_{\rho_1}-\chi_{\rho_2}}$ is supported only at unipotent elements, this character pairing depends only on the values at unipotent elements. But this in turn is independent of $\theta$ by the character formula, Theorem 4.2 in [DL], so

$(\chi_{\rho_1}-\chi_{\rho_2}, \varepsilon_{\textup{\textbf{G}}}\varepsilon_{\textup{\textbf{T}}} R_{\textup{\textbf{T}}}^{\textup{\textbf{G}}}(\theta')) \ne 0.$

for all $\theta'\in\widehat{{\textup{\textbf{T}}}^{F^n}}$.

For any $\theta'\in\widehat{{\textup{\textbf{T}}}^{F^n}}$ in general position ${\varepsilon_{\textup{\textbf{G}}}\varepsilon_{\textup{\textbf{T}}} R_{\textup{\textbf{T}}}^{\textup{\textbf{G}}}(\theta')}$ is the character of an irreducible representation by Theorem 6.8 and Theorem 7.1 in [DL], thus the above shows that it is a constituent of at least one of $\rho_1,\rho_2$. The number of such $\theta'$ is

$\gg (q^n)^{r({\textup{\textbf{G}}})}$

and each $\varepsilon_{\textup{\textbf{G}}}\varepsilon_{\textup{\textbf{T}}} R_{{\textup{\textbf{T}}}}^{\textup{\textbf{G}}}(\theta')$ has dimension

$\varepsilon_{\textup{\textbf{G}}}\varepsilon_{\textup{\textbf{T}}} R_{\textup{\textbf{T}}}^{\textup{\textbf{G}}}(\theta')(1) = |{\textup{\textbf{G}}}^{F^n}/{\textup{\textbf{T}}}^{F^n}|_{p'}\gg \sqrt{|{\textup{\textbf{G}}}^{F^n}|(q^n)^{-r({\textup{\textbf{G}}})}}$

by Theorem 7.1 in [DL]. This proves the lemma.
$\blacksquare$

After finishing with my masters results, I’ll write the first four points as well. Stay tuned!

——————

[DL] P. Deligne, G. Lusztig, Representations of reductive groups over finite fields, Ann. of Math. (2) 103, no. 1, 103?161. (1976).

# Beats Distribution

When another post on Jive Junction’s Facebook group asked about BPMs for lindy hop parties, I decided to check the distribution from recent saved DJed playlists, 14 in total. These include two from big events (Lindy Shock, Vintage Swing Festival), three from local (to Slovenia) events, and the rest are weekly local dance nights in Tel Aviv and Ljubljana. Here’s the plot:

$\mu=146, \sigma=17.1$

Except for the peak in the middle, the distribution is pretty normal. When I DJ, I try to match the speed to what I think will get the most dancers on the floor, in the current song and the next. Over time, I have consciously come to think that 146 is the most ideal BPM for this task – whether after a fast song or a slow song, or even of the same speed, it will bring out the most dancers for this and the next song.

But I haven’t really documented how many people are dancing to each song, and maybe I have it off. So I’ve now started, and I will also randomise my playlists a bit more with the help of a computer, which hopefully is unbiased. I definitely want to try Shana Worel’s suggestion of $\mu=170,\sigma=16.6$ for a few sessions (“with a few outliers sprinkled in for good measure”).

It would be pretty cool to get a result similar to that well known experiment in the west coast world. 122 was it?

————————–

The above plot was made using a python script that I wrote, that goes over playlists in the iTunes Media Library file. This is an xml file, which on my mac was found in ‘~/Music/iTunes/’. You can run the script as well to see your own statistics. If there are playlists that are not relevant, you can name them and the script will disregard them. I used numpy and matplotlib to generate the plot, and those few lines are not included.

from xml.dom.minidom import parse, parseString
from math import sqrt

# Change this to wherever your 'iTunes Music Library.xml' is.
_itunes = &quot;~/Music/iTunes/iTunes Music Library.xml&quot;

# Irrelevant playlists. For example, the bpms from my
# wedding playlist should not be counted.
irrelevant = ['April','From Videos','Wedding!','WeirdThing']

###########################################################
###########################################################

itunes = parse(_itunes)

_tracks = itunes.childNodes[1].childNodes[1].childNodes[24]
_playlists = itunes.childNodes[1].childNodes[1].childNodes[28]

def getKeyIndex(idict, key):
cn = idict.childNodes
for i in range(len(cn)):
if cn[i].hasChildNodes() and cn[i].tagName == u'key' and cn[i].childNodes[0].data == key:
return i
return -1

def getValue(idict, key):
i = getKeyIndex(idict, key)
if i != -1:
cn = idict.childNodes
i = i+1
while not cn[i].hasChildNodes():
i = i+1
if i &gt;= len(cn):
return -1
return cn[i].childNodes[0].data
return -1

# Parse tracks
tracks = {}
for track in _tracks.childNodes:
if track.hasChildNodes() and track.tagName == u'dict':
id = int(track.childNodes[2].childNodes[0].data)
bpm = int(getValue(track, u'BPM'))
tracks[id] = (bpm,)

# Parse playlists
playlists = {}
for playlist in _playlists.childNodes:
badtags = ['Distinguished Kind','Smart Info', 'Folder', 'Master']
name = getValue(playlist, 'Name')
itemsIndex = getKeyIndex(playlist, 'Playlist Items')

if set([getKeyIndex(playlist, tag) for tag in badtags]) == set([-1]) and name != -1 and itemsIndex != -1:
while not playlist.childNodes[itemsIndex].hasChildNodes() or playlist.childNodes[itemsIndex].tagName != u'array':
itemsIndex += 1
if itemsIndex &gt;= len(playlist.childNodes):
break

if itemsIndex &gt;= len(playlist.childNodes):
continue

ptracks = [int(i.childNodes[0].data) for i in playlist.childNodes[itemsIndex].getElementsByTagName('integer')]

playlists[name] = ptracks

for irr in irrelevant:
playlists.pop(irr)

allbpms = []
for p in playlists:
for id in playlists[p]:
if tracks[id][0] &gt; 0:
allbpms.append(tracks[id][0])

mu = (sum(allbpms)+0.0)/len(allbpms)
sigma = sqrt(sum([(x-mu)**2 for x in allbpms])/len(allbpms))

print 'mu:', mu
print 'sigma:', sigma


# Counting Symplectic Matrices Satisfying Something

While working on my thesis, I wanted to bound the dimension of a Jacquet module of a uniform character (i.e. a combination of Deligne-Lusztig characters) using an idea of Gelfand, appearing in ‘Representations of the full linear group over finite fields’ (1970). In modern terms, Gelfand shows that the cuspidal characters of $G=GL_n(\mathbb{F}_q)$ are generic/regular/have a Whittaker model.

For a cuspidal representation $\tau$, and $\psi\in \hat{\mathbb{F}_q}$, a non-trivial additive character, Gelfand shows that (1): $(\textup{res}_U^G\ \tau,\psi)_U$ is a rational function of $q$ of degree $0$, and (2): it equals $1$ for all large enough $q$. Hence it is always $1$.

The first property is proved by a rather explicit computation. Gelfand shows that the number of upper triangular unipotent matrices in a given conjugacy class is a polynomial in $q$, and computes its degree. This is enough, since the character values of a cuspidal character at unipotent elements was already known, due to Green (I think). The second property essentially follows from the first and a multiplicity one result.

I wanted to do the same with a representation of $Sp_{2n}(\mathbb{F}_q)$: show that the dimension of the Jacquet module with respect to some unipotent subgroup was a polynomial and that I can compute its degree. Specifically, I needed to compute the number of matrices such that

$u=\begin{pmatrix} u'&M&Z \\ &I_{2\ell}&M' \\ & &u'^* \end{pmatrix}\in Sp_{2m}(\mathbb{F}_q),$
$\textup{rank}(u-1)=k,$
$u'_{1,2}+u'_{2,3}+...+u'_{m-\ell,m-\ell+1}=c$

for any positive integers $m, k, \ell$, and $c\in\mathbb{F}_q$. (The symplectic form is defined in the linked file below.)

So I did. I knew that I didn’t have a formula for the values of my character at unipotent elements, but I knew that the values are polynomials, and I thought I knew their degrees. This would be enough to prove what I needed. But I was wrong. I did not know the degrees of the polynomials. I couldn’t find a way to compute them without doing some heavy lifting, like in Lusztig’s ‘On the Green Polynomials of Classical Groups’ (1976). Essentially, I was missing the following result: degrees of values at unipotent elements of “small” representations are smaller than degress of values from cuspidal representations. Small here means a semisimple representation that is not also regular. I still don’t know if this is true or not.

But then I found a different way to bound the dimension of my Jacquet module: compute the character’s wave front set and use Lusztig’s results from ‘A unipotent support for irreducible representations’ (1992).

Back to counting matrices. I use the same ideas from Gelfand’s paper, only extending the amount of variables I need to keep track of (not just rank). If you’re interested, here it is:

http://www.math.tau.ac.il/~drorspei/Unipotent Symplectic Matrices Satisfying Something.pdf