text:
Designing safer cities through simulations
Every year 1.25 million people are killed in traffic accidents. In 2016, road crashes resulted in 40,000 deaths and 4.6 million injuries in United States alone. Unity has partnered with the City of Bellevue in Washington State to work towards reducing those numbers through technology.
By using machine learning and simulations, we believe we can identify unsafe intersections and mitigate the risks before the actual accidents happen. In this blog post, we'll talk about the initial steps on this collaboration: we'll introduce the project, talk about the basic computer vision concepts behind its idea and showcase how the Unity Engine is used to create simulated environments to train machine learning models used in the solution.
The initiative
This partnership between Unity and the City of Bellevue is part of a project called Video Analytics towards Vision Zero.
It's a long name, so let's break it down to understand its scope.
- Vision Zero is a multi-national, multi-entity road safety project focused on building a highway system with no fatalities or serious injuries involving road traffic.
- Video Analytics towards Vision Zero is the technology initiative driven by the City of Bellevue to find and fix unsafe intersections by identifying "near miss" situations through video analysis. In other words, to automatically detect would-be accidents that were averted, and modify the intersection to prevent future occurrences.
The overarching goal of the initiative is to create a computer vision system that can leverage cameras spread across the city's many intersections to identify unsafe intersections. By recognizing objects in the video streams – cars, people, bicycles, etc – and their trajectory, the system is expected to provide input to traffic planners about frequency and nature of incidents, allow for reviewing dangerous situations and generate more accurate general reports about those intersections. All this data will allow cities to make sure safety measures are put in place – such as redesigning of crossing lanes, adjustment of stop lights timing, introduction of clearer signage – getting us closer to the the zero fatalities goal aimed by Vision Zero.
This article focuses on the object recognition task, which is the initial piece of the safety system planned for the initiative. Unity Engine's ability to create synthetic representations of the real world is a valuable resource for computer vision applications, as we'll explain ahead.
Training computer vision models
Computer vision is a subfield of artificial intelligence that aims to extract information from images and videos. There has been significant progress in this area for the last five years and it has been widely applied to automatically make sense of real images. In particular, a computer vision model can learn to understand the video feeds from multiple intersections and what goes on around them.
This understanding requires detecting, classifying and tracking different elements of the image. By detecting, we mean locating each object or piece of the scenery in the image. The classification provides information about the type of the detected parts. Additionally, each object is tracked as an individual instance in the image – which is particularly important when dealing with sequences of images or videos. All this metadata allows for the machine learning model to understand the image as a whole.
Usage of real world images is a common approach to provide bootstrapping data for computer vision models. The initial identification and classification metadata is provided through manual work, using specialized tools in a process called annotation.
Here's an example taken from Vision Zero annotation tool:
Notice the bounding boxes created manually by the tool operators around each pedestrian in the street, as well as the "area of interest" defined by the dashed red line.
Once the images are annotated, a part of the set is used for training the machine learning model while a smaller hold-out of the dataset is used to evaluate the performance of the trained model on scenes it has never seen before.
On an abstract level this is how the data is used for training supervised models:
A common challenge with computer vision applications is that finding enough meaningful training and evaluation data is hard. Manual annotation of real world images and videos is the norm and was a big focus on the initial phase of the Video Analytics project. But it is a costly process, and the quality of the labels can be affected by operator fatigue, inconsistencies in the procedures and other human factors.
Even discounting the costs and time, the data obtained through capture is still limited by what the real world can provide. If you need to train your model with some bicycles and buses, wait for them to show up in the same frame. If you want to see what your model does when there's snow, or rain, or fog, then you need to befriend a meteorologist… and be ready to fire off the cameras when they say so.
Simulation is a good way to overcome these limitations. By providing full control of the contents – including full understanding of the nature of each element of the scene – a simulation environment can produce virtually infinite sets of training and evaluation data with absolutely accurate annotations in a multitude of situations that can be either designed for specific cases or generated procedurally to cover as many scenarios as possible.
Scenes and Episodes
Let's go over a couple of concepts at the core of simulations for computer vision: Scenes and Episodes.
Scenes are all the static and dynamic elements, as well as parameters that are modeled in a simulation environment. They include buildings, streets, vegetation (static elements); cars, pedestrians, bicycles (dynamic elements); weather conditions, time of the day (sun position), fog (parameters).
Episodes are determined configurations of the scene elements. For example, a certain placement for pedestrians, a certain route for cars, the presence of rain, road conditions, etc. One can imagine an episode as being an instance of a scene.
In this picture, the boxes on top row represent the individual elements of the scene: static assets, dynamic assets, and parameters – in this case, weather. On the bottom box all pieces from the scene are combined into a complete episode.
When creating simulated data, we typically refer to episode variation as the process of generating different episodes for a given scene. This process can be tailored to create a comprehensive set of situations expected to be found in the real world application of the machine learning model being trained .
Using Unity for real-world simulations
Given the costs and limitations in gathering real-world data, it is natural to consider replacing or augmenting it with synthetic data generated by a game engine such as Unity. Due to recent advances in graphics hardware, rendering, and advent of virtual and augmented reality, the Unity Engine has evolved into a complete 3D modeling tool, able to generate highly photo-realistic simulations. This has been noticed by both industry and academia, and many projects have been developed to take advantage of Unity's simulation capabilities. One of the most recognized projects is SYNTHIA.
Developed entirely on Unity by The Computer Vision Center (CVC) at Universitat Autónoma de Barcelona (UAB), the project focuses on creating a collection of synthetic images and videos depicting street scenes in a diverse range of episode variations.
CVC has been pioneering computer vision research for the past 20 years and its SYNTHIA dataset has become a seminal source for those working on autonomous vehicles perception systems.
For Vision Zero, Unity joined forces once again with CVC to provide the City of Bellevue with the best technology and expertise available.
Through imagery and 3D models provided by the City of Bellevue, and leveraging the integration of Otoy's OctaneRender with Unity Engine, CVC took on creating a set of scenes that can be leveraged to improve both the training and the evaluation of the computer vision models built by Microsoft.
The first simulations
Vision Zero focuses on vehicles and pedestrians interaction on intersections. It means that a proper simulation needs to represent a few city areas with high level of detail, with challenging camera angles, in a multitude of situations – i.e. variation of objects in the scenes – in order to generate the data coverage required by the computer vision models.
The video below shows an intersection (116th Ave NE and NE 12th Street) in the City of Bellevue. The real camera picture at the 16 seconds point is a good baseline to understand the amazing level of photorealism achieved in this simulation.
Not only are the images impressively realistic, but because they are Unity assets, we have all metadata needed for a 100% error-free segmentation – i.e. pixel-level classification of everything in the scene, fundamental data for Computer Vision model training. There's also precise information about distances, depth and materials, eliminating the need for human annotation.
Here's an example of depth metadata, taken from the video above:
The shades of gray represent the distance of each object from the camera, the darker, the closer. The sky has no data and is represented as pure black.
Notice that because we have very fine grained information about the image, it's possible to distinguish individual leaves in the trees and different elements of the buildings facades, for example.
Another snapshot from the video showing the Semantic Segmentation:
Compared to the manual image annotation tool shown earlier in this blog, the difference in quality and precision is clear. Here, it's possible to have pixel-level labeling for full semantic segmentation instead of just a bounding box for coarse object detection. There are many classes of segments, represented in the picture by the different colors. Notice the ability to correctly differentiate between cars and buses, streets and sidewalks. This is powerful metadata that can be leveraged by the model to predict the overlap between different objects in the scene much more accurately than is possible with manually annotated data.
The next steps in our efforts are to start experimenting with the episode variation strategies: different cars, more near-misses, weather variation, etc. and generate a comprehensive dataset to be fed into the training pipeline for the computer vision model. We are also looking at new street intersections to be generated, as we scale out the project.
Initially, the models will target high accuracy on semantic segmentation. Eventually, trajectories will be detected and a complete "near miss" model will be developed to provide the automatic analytics that is the project's goal.
Unity will be supporting Microsoft in the process of evaluating improvements to the computer vision models' performance, tweaking the simulation as needed.
Based on research from different teams – including CVC itself – we expect that the approach of mixing real and simulated data will ensure the best results for the models. We'll be posting concrete results once available.
A future of simulations with Unity
Whereas this project tackles a problem in itself technically complex, its end goal – to improve safety in our cities and save lives – works as a natural catalyst to get all these great teams together. Unity, CVC, Microsoft and City of Bellevue – Industry, Academia and Government working towards a common goal.
It's only natural for Unity to be in the middle of all this, empowering and enabling its partners. After all, it aligns perfectly with in Unity's core values: democratize development, solve hard problems, and enable success.
We have been collecting invaluable knowledge as the project evolves, and those learnings will get incorporated back into the engine so that everybody can benefit.
You can expect Simulation to get even easier and more powerful on Unity as we go, and we are glad that we can do our part to have a safer world in the process.
If you'd like to know more about this project, please contact Jose De Oliveira (firstname.lastname@example.com).
The project is a collaboration with:
Franz Loewenherz, principal transportation planner for the City of Bellevue and head of the Video Analytics initiative
Prof. Antonio M. López, Principal Investigator at the Computer Vision Center (CVC), and Associate Professor of the Computer Science Department, both from the Universitat Autònoma de Barcelona (UAB); as well as Dr. Jose A. Iglesias Research Scientist at CVC.
Ganesh Ananthanarayanan, researcher in the Mobility and Networking group at Microsoft Research.
8 评论订阅评论
Gene`二月 19, 2018 9:02 下午
Really enjoying your Unity ML Agents! How are you using Linux to test and develop ML Agents when it seems all Linux builds of the Unity Editor are affected by an issue that causes instability with .NET 4.6?
Is there a different build I should be using for Linux?
Thomus二月 17, 2018 7:56 上午
Hey admin your sharing way is unique. This is fantastic site.I enjoyed reading your articles. This is truly a great read
Calli二月 14, 2018 11:26 上午
Damn. Its very good.
I mean we can solve most of the city problem related to transportation and roads design by using stimulation games format.
For example, we can build an exact same game copy of a city. then we analyse real problems and then try to solve them in games first. by using games we can save a lot of money which we spent on Hit and trial method for our problem-solving methods. we can see than by expanding width of road what will be result on traffic or what will be effect of traffic lights.
Joel一月 26, 2018 1:33 上午
This is very cool. As hippocoder already said, this could turn into some interesting navmesh agent or other AI improvements of some type, but regardless this is great collaboration and a great learning opportunity. Really appreciate seeing this.
Jack一月 25, 2018 5:05 下午
That's all good but the number one cause of animal and people fatalities and injuries remains overwhelmingly people violating traffic laws and by a huge margin. There is no ambiguity about that at all. So until governments the word over record and analyze publicly available radiation frequencies (sound, sight, and others) to automate ticketing, arrest, and driving license revocation the Vision 0 effect is going to be a miniscule drop in the bucket for massive amounts of taxpayer money.
So what they are going to find is people are blatantly breaking the traffic laws and lying about it. I've been at too many intersections and watched them race through well after the lights had changed.
Alessandro VM Oliveira一月 24, 2018 1:13 上午
Excellent! Great initiative. Thanks for sharing.
hippocoder一月 23, 2018 5:20 下午
There are no drawbacks to this collaboration. I would expect the goal to be experience driven and data driven at this point, but one day would probably see better training for navmesh agents to identify dangers and more (in a game context).
Chris Yarbrough一月 23, 2018 4:41 下午
Thank you for sharing!


text:
Greenmead Historical Park
20501 Newburgh,
Livonia, MI 48152
Livonia, MI 48152
- * What is "Add To Trip"?
First time trying Trip Cart?
Our "Add to Trip" function allows you to save all of the interesting places you find on our site. From this, you can create a full guide that includes all of the places you've selected.
We've partnered with the world's leading road trip planning application, Roadtrippers for this functionality. Once you "Add to Trip" you can do even more by following the prompts to the Roadtrippers application.Learn More
- Phone: (248) 477-7375
Open Year-Round
Although the MEDC uses reasonable efforts to maintain the accuracy of the website content, it is not represented to be error free. Read legal information.

subset: commoncrawl


text:
Government cuts to policing budgets are likely to result in a rise in low-level crime and increased demand on under-pressure local health and education services in parts of Yorkshire, according to one of the region's police commissioners.
In a briefing to West Yorkshire MPs seen by The Yorkshire Post, Mark Burns-Williamson has warned that the expected budgets cuts to be announced later this month will result in fewer local "eyes and ears" on the street to spot potential terrorist threats.
"In the future, we will need to distinguish between what we could do and what we must do to keep the most vulnerable in our county safe."West Yorkshire Police temporary chief constable Dee Collins
The county's police and crime commissioner said his force would be increasingly forced to help out others around the country or deal with national threats, despite losing 39 per cent of its workforce between 2010 and 2020.
He added: "This leaves areas such as West Yorkshire exposed, and in turn, puts its own communities at greater risk."
His claims emerged as Policing Minister Mike Penning apologised in the Commons yesterday after admitting the Government made errors in its controversial reform of the police funding formula.
Mr Penning apologised to MPs and the 43 police forces in England and Wales, adding that initial allocations were "never indicative" of police budgets under the formula and the changes for 2016/17 would now be delayed.
The Yorkshire Post revealed last week that a group of police and crime commissioners (PCCs), including North Yorkshire's, were threatening to take legal action against the Government over funding reforms that could see millions of pounds a year cut from their forces' budgets.
Despite the introduction of new funding criteria being delayed, all police forces still face budget cuts of between 25 and 40 per cent when the Government's comprehensive spending review (CSR) is announced later this month.
Mr Burns-Williamson says Government funding for West Yorkshire will fall by between £107 million and £147 million a year by 2019/20.
This could mean that by 2020 the force, which draws 80 per cent of its funding from central Government as opposed to local tax-payers, employs as few as 3,487 officers, a 39 per cent reduction from the total in 2010.
Mr Burns-Williamson's briefing to MPs last week, written in the hope that they will lobby the Government to change its plans, paints a grimmer picture of the future for policing in West Yorkshire than the Labour PCC's previous public statements.
After revealing in a press release that he had sent out the analysis of "what another round of huge budget cuts could mean for [West Yorkshire Post] and the 2.3 million residents it serves", he declined to release the briefing to The Yorkshire Post. This newspaper has since obtained a copy of the report.
According to the PCC's figures, West Yorkshire Police will lose 13 per cent of its workforce between 2015 and 2018, compared to the national average of six per cent.
It is predicted that the force will have 4,487 officers by the end of March 2016, 1,275 fewer than March 2010. The number of police staff will have fallen by 696 in the same period and the number of PCSOs by 169. In total, more than a fifth of West Yorkshire Police's workforce will have been lost over the six years.
At the same time, the force is seeing big rises in the number of complex 'safeguarding' incidents it has to deal with, such as cyber-crime, child sexual exploitation and human trafficking. The number of human trafficking cases alone is set to more than double in a year, from 35 in 2014/15 to an estimated 87 this financial year.
Mr Burns-Williamson writes in his report: "For West Yorkshire Police managing more complex demand at a time of reducing resources is extremely challenging and has forced a fundamental rethink and alignment of the policing resources available."
The PCC wrote that significant one-off reserves had been ploughed into tackling CSE and cyber-crime, with big investments in telecoms and number plate recognition technology.
He said: "However, there does appear to be a perception, in particular with regard to the larger metropolitan police forces, that further savings can be found. This is simply a false economy in West Yorkshire."
The reduction in numbers combined with increasingly complex cases is said to be putting "severe strain" on staff, with the number of hours lost to sickness absence rising from 2.7 per cent to 3.7 per cent.
Mr Burns-Williamson said neighbourhood policing would not "disappear completely" as it would elsewhere in the country, but that "our community and problem solving neighbourhood activity will be reduced dramatically".
He added: "Our ability to get upstream to prevent crime will be lost, and low level crime and [anti-social behaviour] is likely to increase.
"At a time of severe threat to our national security we need more resources, not less, at the local level to provide the intelligence needed to combat terrorism as the police and PCSOs are increasingly the eyes and ears that identify key threats to our county and to our country.
"The government's announcements about protecting Counter Terrorism funding fail to address this crucial disappearing link and something I have already raised locally and nationally."
The briefing said West Yorkshire Police and other bigger forces would increasingly be asked to help out their smaller counterparts or deal with national threats.
He said: "However, there is a danger that responding to a major national threat may leave even fewer resources at a local level. This leaves areas such as West Yorkshire exposed and, in turn, puts its own communities at greater risk."
Mr Burns-Williamson warned: "Increased demand will inevitably have to be directed to other agencies, NHS, mental health and education, services that the government wants to protect.
"This will fundamentally affect the real improvements made in partnership working with the police across West Yorkshire and most importantly result in a poorer public service to the people who live and work here."
West Yorkshire Police's temporary Chief Constable Dee Collins said that until the CSR announcement was made it was "very difficult to predict with any certainty just how these cuts will impact upon our services and in what areas".
She said: "What we do know, is that we will have to make some extremely difficult decisions, even if the cuts emerge at the lower end of that register.
"We must still find £6.4m over the next two years and this is before we factor in the additional 25 per cent to 40 per cent cuts that are being explored.
"Neighbourhood Policing has always been the very bedrock of our style in West Yorkshire and although it is threatened, we are doing our very best to maintain it.
"The situation remains hugely challenging and with people our main asset, it is reality that we will have to further reduce in size to meet these new savings targets.
"In the future, we will need to distinguish between what we could do and what we must do to keep the most vulnerable in our county safe."
A Home Office spokesman yesterday declined to comment directly on Mr Burns-Williamson's report, but referred instead to policing minister Mike Penning's Commons statement.


text:
Asian transport ministers meeting in South Korea this week will take a major step towards making a decades-old dream a reality -- integrating the entire continent into a single rail network.
Transport ministers and officials from 43 countries will gather in the port city of Busan starting today for this year's ministerial conference on transport organized by the UN Economic and Social Commission for Asia and the Pacific (UNESCAP).
The highlight of the six-day conference will be Friday's signing of the Intergovernmental Agreement on the Trans-Asian Railway (TAR) -- more poetically known as the "Iron Silk Road."
The 81,000km network, first mooted by the UN in 1960, would link capitals, ports and industrial hubs across 28 Asian countries all the way to Europe.
By signing, Asian states will "demonstrate their commitment to working together" on the mammoth project, said Barry Cable, director of UNESCAP's Transport and Tourism Division.
"We expect this will trigger new development in the railway sector both in terms of increasing capacity and in terms of increasing connectivity," he said.
Cable said such an international commitment would make it easier for countries to attract finance for railways from international banks.
"The agreement lays a framework for coordinated development of internationally important rail routes," UNESCAP chief Kim Hak-su said in a statement.
Tardy progress over the past five decades indicates the challenges ahead, despite Friday's scheduled signing. The Cold War, a major obstacle, is over but Asia has its fair share of civil conflicts and tensions.
Other problems include switching between different-gauge tracks, where to stop, how to handle sometimes tricky quarantine and immigration paperwork, and how to safely ferry cargo and people across many borders.
But fast-growing Asia, home to 60 percent of the world's population and generating 26 percent of the world's GDP, deserves better transport, Kim said.
"Asia's demand for efficient transport is greater than at any time in its history," he said.
The Asian continent boasts 13 of the world's top 20 container seaports.
But it has fewer than 100 "dry ports" -- inland container depots -- while Europe has 200 and the US has 370.
The Bangkok-based UN agency believes that the creating of more "ports of international importance" along the Trans-Asian Railway will reduce seaport congestion and transport expenses, as well as promoting trade and employment.
"This could actually stimulate more economic growth of the countries and share the benefits of globalization with a wider part of the community," Cable said, citing uneven development between coastal and inland areas.
Twelve of the world's 30 landlocked countries are in Asia, he said.
The TAR network would connect the vast existing railway lines across Asia with Russia and Mongolia in the north, Malaysia and the Philippines in the south, South Korea in the east and Iran and Turkey in the west.
At the Busan conference, delegates will also issue two other declarations -- one on the direction of regional transport development in the next five years and the other on road safety.
UNESCAP estimates that last year 440,000 people died and at least 2 million were injured on Asia-Pacific roads.
This is around half of the world's fatalities, even though only one-fifth of the world's motor vehicles are registered in the region.


text:
Award-winning Professional Photographer, Rich Turner is known and applauded for his long-term commitment to documenting the California Delta; he has been photographing the area for over 30 years. He has captured the Delta in both color and black & white images and has sought out and memorialized the diversity of the Delta region in all its breath taking beauty.
At this critical point in the history of the Delta, it is Turner's goal to celebrate the extraordinary grandeur of its ecosystems, wild life, agriculture, communities and recreational pleasures before they change forever.
Commentary will be provided by Christina Swanson, PhD the Executive Director & Senior Scientist of The Bay Institute.