Home » Day-to-day
Category Archives: Day-to-day
Speaker: Stephen Altschul
Title: Dirichlet Mixtures, the Dirichlet Process, and the Topography of Amino Acid Multinomial Space
Venue: Tuesday May 23rd 3.30 PM Department of Statistics, Lecture Theatre (Lower Ground)
Abstract: The Dirichlet Process is used to estimate probability distributionsthat are mixtures of an unknown and unbounded number of components.Amino acid frequencies at homologous positions within related proteins have been fruitfully modeled by Dirichlet mixtures, and we have used the Dirichlet Process to construct such distributions. The resulting mixtures describe multiple alignment data substantially better than do those previously derived. They consist of over 500 components, in contrast to fewer than 40 previously, and provide a novel perspective on protein structure. Individual protein positions should be seen not as falling into one of several categories, but rather as arrayed near probability ridges winding through amino-acid multinomial space.
The slides will be made available after the talk.
Comment: Stephen Altschul has finally proven that I can’t add 2 and 2. I have attended Altschul Dinners at my College [University College, Oxford] and never thought of connecting the two words Altschul and Altschul despite their obvious similarity. It is in honour of Stephen’s grandmother, whose brother was Arthur Lehman Goodhart and Master of UNIV 1951–63.
Today we hosted a talk by Gereon Kaiping from Leiden University. The talk went through the pipeline his group uses to go from linguistic data collected in the field to reconstructed phylogenetic trees of languages produced using mostly off-the-shelf tools. There was a lively discussion with an audience of linguists, bioinformaticians, and statisticians.
Gereon has kindly made his slides available, which can be viewed below, or downloaded.
There is a famous danish sketch called “Jarl Kakadue” from the show “Casper og Mandrilaftalen”. In the sketch, Jarl explains how he completed an iron man, but instead of running a marathon, he got a good nights sleep instead.
“But isn’t that cheating?” to host asks, to which Jarl replies “No, because such a run takes a couple of hours, but a proper nights sleep is at least 8 hours.”
As the sketch goes on, more and more of the exercise gets replaced. The full thing can be seen here: (in danish)
The concept of Extreme Reading is also a modified iron man in the following sense:
instead of swimming, we read a book.
instead of cycling, we summarise the book
instead of running a marathon, we run half a marathon (over 3 days)
So each day, we read for a couple of hours, ran 7 kilometers, read some more and then we summarized the book for each other and discussed it.
The book i question was “The origin and nature of life on earth – the emergence of the fourth biosphere” – by Eric Smith and Harold J. Morowitz
Unfortunately, the book is rather wordy and not very mathematical. The individual sections are nicely structured, but the book lacks an main message and sense of direction.
This is puzzling, since Morowitz other books are usually shorter and more precise. However, Morowitz died before the book was published, was very weak the last decade, published little in that period and was in general very short in his formulations, while this book is very long (at times lenghty). It is unclear how much Morowitz contributed to the present book.
This book is 600 pages long and consists of 8 chapters. This is a very hard topic to write a coherent book about and the chapters are quite free-standing contributions to describing or explaining the theory of life.
Eric Smith gave a talk somewhat based on the book, which can be found here: https://www.youtube.com/watch?v=0cwvj0XBKlE
The 4 geospheres are:
The point of the title is that life should be though of as a planetary property. However, the point seems more philosophical than scientific, which is the case with many of the subtle points in the book.
A longer summary will be added later.
Overall, the project was a success. We managed to run and read a lot. It is a very satisfying feeling to be both mentally and physically exhausted and we can definitely recommend similar undertakings.
Tomorrow afternoon we are hosting a talk by Gereon Kaiping, who we met at a recent workshop. All are welcome; details below.
Time and location: Department of Statistics on Tuesday 25th April at 4.00 pm – 5.00 pm in the Small Lecture Theatre (LG.03).
Speaker: Gereon Kaiping , University of Leiden
Title: Some Assembly Required: From sounds to histories in 8 steps using mostly off-the-shelf tools.
Abstract: Phylogenetic methods are gaining traction in linguistics, but have so far been quite inaccessible to linguists:
The core tools doing the tree construction – whether they be heuristic or Bayesian – often come from bioinformatics, and their inputs (eg. Nexus files) and outputs (eg. Newick trees without explicit reconstruction) conform to biological, not linguistic standards – or they are ad-hoc written for a specific datasets. However, this situation is changing: In this talk, I will present a collection of tools, most of which are published elsewhere, that together go the full way from linguistic fieldwork via public cross-linguistic linked databases and Bayesian inference tools to plots of phylogenetic trees with ancestral state reconstruction. I will describe both emerging standards in quantitative historical linguistics that make this process easier, and specific challenges that arose in the construction of this tool chain. The talk will conclude with the discussion of some results from the reconstructed word-meaning correspondences in the Lesser Sunda region of Indonesia, and how they feed back into improving our data and understanding of the local language history.
Sadly the workshop is over, and we are preparing to return to sunny Oxford! We enjoyed two final talks today, which we summarise below. We have also written up summaries of Tuesday’s talks, and Wednesday’s talks.
Causal inference of evolutionary networks – Johannes Dellert, University of Tübingen
This speaker began by discussing the difficulties with building up phylogenetic networks. Most phylogenetic methods (on languages as well as in biological contexts) are based on trees, but these trees imply a greater independence than we know to be realistic – they usually fail to capture language contact and influence, which can be a major driver of similarity between languages (separate from inheritance). Methods which do utilise networks are usually either visualisations of other kinds of data (where nodes don’t correspond to languages, for instance), or are restricted to narrow sub-classes of network structure which are not often powerful enough to capture the kinds of relationships that one would like to capture.
To address this, the speaker presented a project based on the concept of causal inference, building a network of causal relationships between observational data alone. Correlation does not imply causation – but by considering correlations on a connected network, it’s possible to delete edges on the network in such a way that leads to a structure of causal relationships explaining the observed correlations. The results were mostly very good, and went beyond any previously available method or tool for such analysis. There are some artefacts, e.g. with a group of languages that had influence from German, but where one language in particular had had a lot of German influence and it appeared that this language then had influence on the others (rather than all from German), but overall it seems like a very promising project with great results and an inspiringly creative and successful approach to a very difficult problem.
Simulating lexical evolution with semantic shifts – Gereon Kaiping (*) and Johann-Mattis List (^), University of Leiden (*), Max Planck Institute for the Science of Human History (^)
This talk began with a discussion of some of the problems with current quantitative methods in historical linguistics. A major such problem is the lack of proper data on historical language change, leading to a trend towards models not being properly validated and tested. There is also not much simulation done to test methods, and most existing simulations tend to be very simple. This project aims to develop a more realistic model of language change, under which simulations might be done which could lead to better validation and testing of other quantitative historical linguistic methods. The model further considers semantic drift and replacement, in contrast to most previous methods which consider cognates only corresponding to the same concepts.
This built on concepts from Saussure about the form and meaning of words being ‘two sides of the same coin’. The model sees a language as a bipartite graph between a network of concepts and a vector of words. The evolution of the model involves updating the weighting of edges between the concepts and the words, corresponding to the changing set of vocabulary and meanings of words, over a phylogenetic tree. This draws on game theoretic ideas. They also presented some validation and parameterisation of their models based on available data sets. Their software is open source and available online: https://github.com/anaphory/simuling
Further evidence for punctuated language evolution – Gerhard Jäger, University of Tübingen
This talk discussed the concept of punctuated evolution – that is, evolution where the most active phase of change happens just after speciation takes place. In biology this has been suggested as an explanation for the relatively few ‘intermediate stage’ fossils that are found – it seems that it’s often the case that a species arises, quickly evolves into a relatively stable state, and stays fairly unchanged for some time. It has been suggested that the same phenomenon might occur in language change (e.g. by Dixon in 1997).
Two methods had been reproduced: one from Atkinson et al (2008) which works on manually labelled lists of cognate pairs, and one from Holman and Wichmann (2016) which uses language distance (without needing labelled cognate data). Overall the study’s results seemed to suggest that punctuated evolution may indeed be taking place to some extent in language change.
Building histories of Slavic on parallel texts – Ruprecht von Waldenfels, University of Zurich
This talk was quite different from most others at the workshop in two main ways: it examined a language family history which is known in some detail already, and the methods revolved around the use use of parallel texts rather than word lists or other data.
Taking texts which have been translated into all of the languages considered, the study looked at different language features individually, finding different connections between languages. Since the history of the language family is fairly well known, the speaker was able to explain the nature and history of these different relationships for individual language features. This seemed a step forward for Slavic language studies, confirming much more manual work with much automated analysis. It also sent a strong message to those in the audience, that many methods in use (e.g. based on language similarity) may induce a history, but in fact there can be many histories behind the relationships between languages in any particular group.
Reconstructing language ancestry by performing word prediction – Peter Dekker, University of Amsterdam
This talk described a project based on the use of recurrent neural networks with an encoder-decoder structure to detect cognates, in a supervised machine learning framework. This process has some analogy to problems in machine translation, where neural network approaches have been applied with some success, and this project draws on some of the progress in that field to solve problems here. The neural network is trained on pairs of words corresponding to the same concept in different languages. Since this method avoids relying on manual labelling of cognate and non-cognate pairs, the goal is to take all in but only really learn from the cognate pairs, which is achieved by the design of the loss function. Overall it seemed like the project was reaching a baseline of success in line with existing models, and that it had promising scope for tweaks to improve its performance further.
Sound change phylogeny in Uralic family trees and networks – Jyri J. F. Lehtinen, University of Helsinki
This talk began by acknowledging some of the criticisms that have been aimed at the use of phylogenetic methods in historical linguistics in general by linguists. A primary such complaint was the concept of “garbage in, garbage out”. The speaker described a study which involved a very careful process of data selection. The study looked at shared innovations in Uralic languages by looking at reconstructed protoforms and attested forms of words – taking only the words with the most reliable, stable, and regular reconstructed protoforms known from the literature (taking care to avoid including data which has been superseded or isn’t considered reliable).
The study focussed on phonological data, and compared the results of phylogeny reconstruction with this data to other studies using lexical data, as well as trees constructed from qualitative approaches. The results seemed very positive for the approach.
Deep learning and historical linguistics: two case studies – Taraka Rama, University of Tübingen
As well as a high-level introduction to neural networks, this talk discussed the use of neural networks for two linguistics applications: cognate identification and dialect classification. For cognate identification, convolution neural networks are used. This doesn’t require explicit character alignment, and the network was designed with a structure which allows word relatedness and language relatedness, which both inform cognate inference, to be simultaneously learned. The results were positive, even with relatively small data sizes.
For dialect classification, an unsupervised learning approach was taken. Autoencoders were trained on large numbers of words encoded in IPA format, without the need of explicit manual alignment and cognacy judgments. The approach produced some interesting and good-looking maps of dialect distribution in a few different countries.
Tracking modern human population history from linguistic and cranial phenotype – Hugo Reyes-Centeno, University of Tübingen
This talk took a very creative approach to address a conjecture first raised by Darwin – essentially over how much human genealogy can tell us about language genealogy. To examine this relationship, the study made use of the “serial founder effect”, which essentially says that there is less genetic diversity seen as the population moves further from its starting point (as each time a new population is established, it is drawn from only some fraction of the previous, larger population, so the gene pool of the new population is based on a subset of the original genepool). The study investigated whether there’s a relationship between linguistic diversity and genetic diversity.
Properties of cranial bone fragments were used as a phenotypical proxy for genotypic data. By comparing skull fragments from various regions with language diversity from those reasons, the relationship between them was studied. They also controlled for the effects of geography, in terms of distance from the widely-accepted origin population of humanity in Africa. Overall there was not a significant statistical signal of a relationship, but the speaker discussed further aspects of the serial founder affect which could be investigated to get a more detailed picture.
Today we (Jotun and Jimi) attended the second day of the Phylogenetic Methods in Historical Linguistics workshop in Tübingen. We heard several interesting talks on a range of topics, including one from Jotun himself. Here’s a summary of what we heard about today.
Introduction to comparative biology and statistical alignment – Jotun Hein, University of Oxford
Jotun’s talk introduced some fundamental ideas in comparative biology, going into some detail of basic statistical alignment techniques, and summarising some potential applications to linguistics via projects done in the Hein group and at Oxford more widely.
He discussed the significance of the different possible structures that can be used to model data in these settings. Michael Golden’s work on protein structure evolution was an example of the use of more complex data structures than models which operate on plain strings or sequences. Jotun mentioned Jimi Cullen’s work on context-dependent sequence evolution and mentioned Luke Kelly’s impressive work with Geoff Nicholls on lateral transfer in stochastic Dollo models.
There was a widely enthusiastic reaction to Jotun’s mention of his brief work with Markus Gerstel on transforming English grammar into German through the use of treebank data – this idea could be worth reviving!
What can shared structures tell us about linguistic similarity? – Andrea Fischer, Universität des Saarlandes
This talk was about the use of ideas from information theory to measure similarity between languages. By finding sequential correspondences between parts of similar words in different languages, it’s possible to compress the list of words by defining rules determining the relationships between letters or groups of letters in words in different languages. Using MDL (minimum description length), it’s possible to get an idea of how close two languages are, in a measure roughly inspired by how easy it might be for a speaker of one language to read text in the other language. The process also infers correspondences between parts of words, which in certain circumstances correspond strongly to known (or sensible) linguistics correspondences.
This seems like a rich work developed in a powerfully modular way, making use of a variety of techniques and ideas from information theory, making it quite extensible and modifiable.
Genes, speech, and language – Dan Dediu, Max Planck Institute for Psycholinguistics
This speaker strove to convince us of the interesting connections between genetics and language. The first example he gave was to do with hearing and deafness. He pointed out several specific genetic mutations which (sometimes surprisingly) lead to deafness. One such mutation has led to higher rates of deafness in some communities where it’s very common, and this in turn has led to the development of sign languages in those communities. As the speaker pointed out, this is a compelling language example of genetic-cultural co-evolution.
He went on to describe his work on identifying links between anatomical vocal variation and language variation. This was focussed on a study of languages which include certain types of “click” sounds, based on the hypothesis that such languages are more likely to develop in communities where people typically have a very small alveolar ridge. He has conducted experiments that seem to indicate that this does indeed make it easier to produce these sounds, and that it does seem to be a common feature of the mouths of people from some areas where click-languages arose. He gave the impression that this is the tip of the iceburg, and that there’s a lot more work to be done before any major conclusions can be drawn, but it seems like a promising line of study.
Constructing language phylogenies on different kinds of data – Stephan Eekman, University of Amsterdam
The speaker went through some phylogenies of North Germanic languages that he had generated using standard tools, by varying the type of input data. He used lexical data (which has been most widely used for inferring language phylogenies in the past), phonological data, and morphosyntactic data, as well as running his models on a dataset combining all of these types. He used the standard Swadesh word list, as well as a word list he had constructed himself of vocabulary relating to domestic animals (apparently inspired by a sheep he saw out the window while working on the project!).
He was surprised to find that the best performance by far seemed to be those models run on the domestic animals data. He discussed some aspects of this type of vocabulary which might have contributed to these promising results. Overall the talk raised things to think about when designing future studies more than delivering any major conclusions of its own.
Applying three evolutionary models to linguistics – Andrew Meade, University of Reading
This wide-ranging talk covered three main topics. The first of these was heterogeneous rates of change on a phylogenetic tree, with examples of animal size change in biology and language change with migration in linguistics. The second topic was on phonemic phylogenetic influence, and applying the concept of concerted evolution to identify regular sound changes in language change. The third was applying population genetics methods to word use – exploring how it comes to be that there are many widely-used words for “sofa” but essentially just one for “axle”.
This talk was a strong demonstration of the potential for these methods from comparative biology in their applications to linguistics, and suggest that the speaker has generated quite a wide-reaching body of work with these approaches.