For March's University of Nottingham Public Lecture Series talk, Dr Mandy Roshier, from the School of Veterinary Medicine and Science, and Dr Steve North, from the School of Computer Science, join forces to talk about Bits And Bytes - When Horses Meet Computers. @Gav Squires was there and has kindly written this guest post summarising the event, with some linkage added by NSB.
Mandy and Steve are looking at how horses feel and what they want, using computers to measure their behaviour in an objective manner so that we can improve our understanding and the horses' welfare.
The research has focused on animal-computer interaction. Animals have interacted with our technology throughout human history and they now interact with our computer-based systems, whether they know it or not. The lack of an animal perspective on system design can have a negative effect on both animal users and the purpose for which the technology was developed. Animal-computer interaction is a recent field of study and is still very small.
However, horses have been interacting with our technology for a long time. Before 4000BCE, horses were only used as food for humans. Then between 4000-3000BCE, humans started to use horses for traction and transport. While we still ate horses during this period, this was the beginning of horses having to interact with our technology. These were "hard" technologies, the saddle, bridle, reigns, bit, halter, whip, working collar, harness, chariot, cart and plough. Then there was the most significant development - the "hot shod" horse shoe. As the horse enable humans to travel great distances, trade, carry cargo and share culture and language, it could be argued that the horse was a primary driver of human technological development.
Horse-computer interaction can include environmental, physical training, health and sport performance analysis and more general health such as pregnancy monitors and web cams. Some of these techniques can be invasive. However, touch screen computers are changing the way that scientists carry out equine research as they can take out the risk of human influence in equine decision making and actions. There is also the Aktiv stable, which controls a horse's environment. Microchips attached to the horse control feeding, watering and social interaction between horses. There are benefits to the horse in that it allows a more natural lifestyle but it reduces human interaction.
How do horses respond to their environment? How does this differ between sports horses, those just kept in regular stables and feral horses? (there are no such thing as wild horses, only feral) Behaviour is a part of welfare, although it is one that vets can often overlook in favour of the physiological side of animal care. In the 70s, the Animal Welfare Council came up with their five freedoms, which included a reference to behaviour:
* Freedom from hunger and thirst
* Freedom from discomfort
* Freedom from pain, injury and disease
* Freedom to express normal behaviour
* Freedom from fear and distress
Animal research can include psychology (the study of the mind), ethology (the study of behaviour) and physiology (for example measuring something like cortisol levels) The physiology and the ethology need to be considered together so that you can tell whether a rise in cortisol levels is due to increased stress or increased excitement for example. Within these three areas, you can record observable actions and interpret them. Computers help here as they can cope with large levels of data. This has led to research such as EquiFACS, which looks at equine anatomy by recording minute movements of muscle. This initially started by looking at humans but has moved on to other animals. Similarly, the Horse Grimace Scale was born from research that started with rodents but now looks at a way of measuring how much pain a horse is in. Meanwhile equine motion has fascinated us for years, from the earliest cave paintings to the present day.
Following behaviour is when horses follow each other in a group. There are a number of instances of this:
* Following as reproductive behaviour between a mare and a foal, parenting, early development
* Following as intermale interaction
* Trek as maintenance behaviour - locomotion
* Parallel prance as intermale behaviour
* Chase as either intermale behaviour or as play
* Fleeing in response to a threat or unfamiliar stimulus - stampede
Why is this of interest? If we can learn the intricacies of horse-horse following and horse-human following, it could improve the training of horses. How is movement initiated when horse follows horse? When horses follow and mirror the speed and direction of humans, does the horse think that the human is another horse? Is this imprinting or is it learned?
Mandy and Steve are working on HABIT - the Horse Automated Behaviour Identification Tool. This is considering the use of technology to automatically recognise horse behaviours. The goal is to automatically produce YouTube quality videos of automatic analysis of horse-horse and horse-human behaviours. From this it should be possible to asses equine behaviour from a welfare perspective and answer the questions, "Are horse interacting with humans as if they were other horses?" and "Are horse in training behaving 'normally'?" From here the programme aims to inform, increase people's knowledge base and aid training. There is a focus on low stress handling, welfare and safety and a consideration for species interactions.
There are five bedrocks of HABIT
1.Know your species. Where did it come from originally? How did it evolve? How was it domesticated? What is its social life like?
2.Sensory capabilities. How do they view the world? Humans and horses see colour differently because humans are tri-chromatic but horses are only b-chromatic. Horses also have very mobile ears
3.Communication. Reading body language - what do these expressions mean?
4.Our verbal and body language. What's in a name? Our words can make a difference in how others interact with a horse
5.Consider the individual, both nature and nurture
Using computer vision and machine learning, the system performs video based behaviour identification. It features helper apps that provide a smooth workflow and which processes longer videos into shorter clips ready for analysis by the main system. Computerised tracking and reliability is difficult in the field, as is automatically identifying different horses.
Computer vision can be inflexible. Real time processing can also cause some issues compared to retrospective processing. Some simple behaviours can be processed in real time but more complex behaviours need to be processed. Longer videos are processed into short ones by first identifying segments that contain horses. The video is then summarised in a similar way to instant highlights in sport. The hardest thing to do is to identify those sections that contain horse action and this happens frame by frame using Haar Cascades.
Haar Cascades were invented by Viola and Jones in 2002, named after Alfred Haar for his work in the 20th century on wavelets. It's a machine learning approach where software is trained from many positive and negative images and then used to detect objects in other images. It combines increasingly more complex classifiers into a cascade. It's used for face recognition, finger print recognition and number plate detection on motorways.
To train the system, the computer moves each member of a set of graphical shapes, called features, across the test image at different scales and orientations. Each of these features consists of contrasting regions of black and white rectangles. At each position, the code checks if the image contains a similar range of contrast to the current feature. The best features in a specific position, scale, etc are retained as useful classifiers. The best classifiers are separated in ever more complex stages, these sequential stages are the cascade. A test image can be rejected at an early stage, using a small number of classifiers, thus saving computing power.
For an efficient detector for HABIT, there needs to be unique features of horses that are readily identifiable and visible from multiple viewpoints. The horse "ear detector" has proven to be quite efficient and when this is combined with detectors for "legs" and "side view", reliability increases.
Next, we need to analyse the clips for behaviours. In the field of computer vision, this is sometimes called action spotting. Firstly, you train a classifier to decide if a behaviour is present in the test video clip. Then you repeat for all horse behaviours, of which there are lots. Then you can test a video clip with all of the classifiers and report one or more behaviours identified. To train the classifier, you have to extract key frames to build up a bag of visual words. From here histograms are built that enable you to identify examples of the behaviour that you're looking for.
The next stages of the research are continuing the development of the HABIT system, especially the behaviour identification module. Then work needs to be done on collecting a video of specific behaviours for training and testing. From there, it might be possible to start applying HABIT-like approaches to behaviour identification in other species.
The Public Lecture Series returns on the 20th of April at 6pm where Professor Philip Moriarty will talk about When the Quantum Uncertainty Principle Goes Up to 11. For more information, visit the Public Lecture Series website: http://www.nottingham.ac.uk/physics/outreach/science-public-lectures.aspx
Image Sources
All via Gav Squires
Mandy and Steve are looking at how horses feel and what they want, using computers to measure their behaviour in an objective manner so that we can improve our understanding and the horses' welfare.
The research has focused on animal-computer interaction. Animals have interacted with our technology throughout human history and they now interact with our computer-based systems, whether they know it or not. The lack of an animal perspective on system design can have a negative effect on both animal users and the purpose for which the technology was developed. Animal-computer interaction is a recent field of study and is still very small.
 |
| Prehistoric and more recent attempts at depicting horse motion |
However, horses have been interacting with our technology for a long time. Before 4000BCE, horses were only used as food for humans. Then between 4000-3000BCE, humans started to use horses for traction and transport. While we still ate horses during this period, this was the beginning of horses having to interact with our technology. These were "hard" technologies, the saddle, bridle, reigns, bit, halter, whip, working collar, harness, chariot, cart and plough. Then there was the most significant development - the "hot shod" horse shoe. As the horse enable humans to travel great distances, trade, carry cargo and share culture and language, it could be argued that the horse was a primary driver of human technological development.
Horse-computer interaction can include environmental, physical training, health and sport performance analysis and more general health such as pregnancy monitors and web cams. Some of these techniques can be invasive. However, touch screen computers are changing the way that scientists carry out equine research as they can take out the risk of human influence in equine decision making and actions. There is also the Aktiv stable, which controls a horse's environment. Microchips attached to the horse control feeding, watering and social interaction between horses. There are benefits to the horse in that it allows a more natural lifestyle but it reduces human interaction.
How do horses respond to their environment? How does this differ between sports horses, those just kept in regular stables and feral horses? (there are no such thing as wild horses, only feral) Behaviour is a part of welfare, although it is one that vets can often overlook in favour of the physiological side of animal care. In the 70s, the Animal Welfare Council came up with their five freedoms, which included a reference to behaviour:
* Freedom from hunger and thirst
* Freedom from discomfort
* Freedom from pain, injury and disease
* Freedom to express normal behaviour
* Freedom from fear and distress
Animal research can include psychology (the study of the mind), ethology (the study of behaviour) and physiology (for example measuring something like cortisol levels) The physiology and the ethology need to be considered together so that you can tell whether a rise in cortisol levels is due to increased stress or increased excitement for example. Within these three areas, you can record observable actions and interpret them. Computers help here as they can cope with large levels of data. This has led to research such as EquiFACS, which looks at equine anatomy by recording minute movements of muscle. This initially started by looking at humans but has moved on to other animals. Similarly, the Horse Grimace Scale was born from research that started with rodents but now looks at a way of measuring how much pain a horse is in. Meanwhile equine motion has fascinated us for years, from the earliest cave paintings to the present day.
 |
| Previously, there have only been cumbersome way of analysing horse motion |
Following behaviour is when horses follow each other in a group. There are a number of instances of this:
* Following as reproductive behaviour between a mare and a foal, parenting, early development
* Following as intermale interaction
* Trek as maintenance behaviour - locomotion
* Parallel prance as intermale behaviour
* Chase as either intermale behaviour or as play
* Fleeing in response to a threat or unfamiliar stimulus - stampede
Why is this of interest? If we can learn the intricacies of horse-horse following and horse-human following, it could improve the training of horses. How is movement initiated when horse follows horse? When horses follow and mirror the speed and direction of humans, does the horse think that the human is another horse? Is this imprinting or is it learned?
Mandy and Steve are working on HABIT - the Horse Automated Behaviour Identification Tool. This is considering the use of technology to automatically recognise horse behaviours. The goal is to automatically produce YouTube quality videos of automatic analysis of horse-horse and horse-human behaviours. From this it should be possible to asses equine behaviour from a welfare perspective and answer the questions, "Are horse interacting with humans as if they were other horses?" and "Are horse in training behaving 'normally'?" From here the programme aims to inform, increase people's knowledge base and aid training. There is a focus on low stress handling, welfare and safety and a consideration for species interactions.
There are five bedrocks of HABIT
1.Know your species. Where did it come from originally? How did it evolve? How was it domesticated? What is its social life like?
2.Sensory capabilities. How do they view the world? Humans and horses see colour differently because humans are tri-chromatic but horses are only b-chromatic. Horses also have very mobile ears
3.Communication. Reading body language - what do these expressions mean?
4.Our verbal and body language. What's in a name? Our words can make a difference in how others interact with a horse
5.Consider the individual, both nature and nurture
 |
| There are many breeds of horse |
Using computer vision and machine learning, the system performs video based behaviour identification. It features helper apps that provide a smooth workflow and which processes longer videos into shorter clips ready for analysis by the main system. Computerised tracking and reliability is difficult in the field, as is automatically identifying different horses.
Computer vision can be inflexible. Real time processing can also cause some issues compared to retrospective processing. Some simple behaviours can be processed in real time but more complex behaviours need to be processed. Longer videos are processed into short ones by first identifying segments that contain horses. The video is then summarised in a similar way to instant highlights in sport. The hardest thing to do is to identify those sections that contain horse action and this happens frame by frame using Haar Cascades.
Haar Cascades were invented by Viola and Jones in 2002, named after Alfred Haar for his work in the 20th century on wavelets. It's a machine learning approach where software is trained from many positive and negative images and then used to detect objects in other images. It combines increasingly more complex classifiers into a cascade. It's used for face recognition, finger print recognition and number plate detection on motorways.
 |
| HAAR like features used in image processing |
To train the system, the computer moves each member of a set of graphical shapes, called features, across the test image at different scales and orientations. Each of these features consists of contrasting regions of black and white rectangles. At each position, the code checks if the image contains a similar range of contrast to the current feature. The best features in a specific position, scale, etc are retained as useful classifiers. The best classifiers are separated in ever more complex stages, these sequential stages are the cascade. A test image can be rejected at an early stage, using a small number of classifiers, thus saving computing power.
For an efficient detector for HABIT, there needs to be unique features of horses that are readily identifiable and visible from multiple viewpoints. The horse "ear detector" has proven to be quite efficient and when this is combined with detectors for "legs" and "side view", reliability increases.
 |
| The Habit Ear Detector, detecting ears. |
Next, we need to analyse the clips for behaviours. In the field of computer vision, this is sometimes called action spotting. Firstly, you train a classifier to decide if a behaviour is present in the test video clip. Then you repeat for all horse behaviours, of which there are lots. Then you can test a video clip with all of the classifiers and report one or more behaviours identified. To train the classifier, you have to extract key frames to build up a bag of visual words. From here histograms are built that enable you to identify examples of the behaviour that you're looking for.
The next stages of the research are continuing the development of the HABIT system, especially the behaviour identification module. Then work needs to be done on collecting a video of specific behaviours for training and testing. From there, it might be possible to start applying HABIT-like approaches to behaviour identification in other species.
The Public Lecture Series returns on the 20th of April at 6pm where Professor Philip Moriarty will talk about When the Quantum Uncertainty Principle Goes Up to 11. For more information, visit the Public Lecture Series website: http://www.nottingham.ac.uk/physics/outreach/science-public-lectures.aspx
Image Sources
All via Gav Squires
Great blog thanks
ReplyDelete