A.I. Between Video Frames

Machine learning system efficiently recognizes activities by observing how objects change in only a few key frames
From Massachusetts Institute of Technology

September 13, 2018 -- Given only a few frames of a video, humans can usually surmise what is happening and will happen on screen. If we see an early frame of stacked cans, a middle frame with a finger at the stack's base, and a late frame showing the cans toppled over, we can guess that the finger knocked down the cans. Computers, however, struggle with this concept.

In a paper being presented at this week's European Conference on Computer Vision, MIT researchers describe an add-on module that helps artificial intelligence systems called convolutional neural networks, or CNNs, to fill in the gaps between video frames to greatly improve the network's activity recognition.

The researchers' module, called Temporal Relation Network (TRN), learns how objects change in a video at different times. It does so by analyzing a few key frames depicting an activity at different stages of the video -- such as stacked objects that are then knocked down. Using the same process, it can then recognize the same type of activity in a new video.

In experiments, the module outperformed existing models by a large margin in recognizing hundreds of basic activities, such as poking objects to make them fall, tossing something in the air, and giving a thumbs-up. It also more accurately predicted what will happen next in a video -- showing, for example, two hands making a small tear in a sheet of paper -- given only a small number of early frames.

One day, the module could be used to help robots better understand what's going on around them.

"We built an artificial intelligence system to recognize the transformation of objects, rather than appearance of objects," says Bolei Zhou, a former PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL) who is now an assistant professor of computer science at the Chinese University of Hong Kong. "The system doesn't go through all the frames -- it picks up key frames and, using the temporal relation of frames, recognize what's going on. That improves the efficiency of the system and makes it run in real-time accurately."

Co-authors on the paper are CSAIL principal investigator Antonio Torralba, who is also a professor in the Department of Electrical Engineering and Computer Science; CSAIL Principal Research Scientist Aude Oliva; and CSAIL Research Assistant Alex Andonian.

Picking up key frames

Two common CNN modules being used for activity recognition today suffer from efficiency and accuracy drawbacks. One model is accurate but must analyze each video frame before making a prediction, which is computationally expensive and slow. The other type, called two-stream network, is less accurate but more efficient. It uses one stream to extract features of one video frame, and then merges the results with "optical flows," a stream of extracted information about the movement of each pixel. Optical flows are also computationally expensive to extract, so the model still isn't that efficient.

"We wanted something that works in between those two models -- getting efficiency and accuracy," Zhou says.

The researchers trained and tested their module on three crowdsourced datasets of short videos of various performed activities. The first dataset, called Something-Something, built by the company TwentyBN, has more than 200,000 videos in 174 action categories, such as poking an object so it falls over or lifting an object. The second dataset, Jester, contains nearly 150,000 videos with 27 different hand gestures, such as giving a thumbs-up or swiping left. The third, Charades, built by Carnegie Mellon University researchers, has nearly 10,000 videos of 157 categorized activities, such as carrying a bike or playing basketball.

When given a video file, the researchers' module simultaneously processes ordered frames -- in groups of two, three, and four -- spaced some time apart. Then it quickly assigns a probably that the object's transformation across those frames matches a specific activity class. For instance, if it processes two frames, where the later frame shows an object at the bottom of the screen and the earlier shows the object at the top, it will assign a high probability to the activity class, "moving object down." If a third frame shows the object in the middle of the screen, that probability increases even more, and so on. From this, it learns object-transformation features in frames that most represent a certain class of activity.

Recognizing and forecasting activities

In testing, a CNN equipped with the new module accurately recognized many activities using two frames, but the accuracy increased by sampling more frames. For Jester, the module achieved top accuracy of 95 percent in activity recognition, beating out several existing models.

It even guessed right on ambiguous classifications: Something-Something, for instance, included actions such as "pretending to open a book" versus "opening a book." To discern between the two, the module just sampled a few more key frames, which revealed, for instance, a hand near a book in an early frame, then on the book, then moved away from the book in a later frame.

Some other activity-recognition models also process key frames but don't consider temporal relationships in frames, which reduces their accuracy. The researchers report that their TRN module nearly doubles in accuracy over those key-frame models in certain tests.

The module also outperformed models on forecasting an activity, given limited frames. After processing the first 25 percent of frames, the module achieved accuracy several percentage points higher than a baseline model. With 50 percent of the frames, it achieved 10 to 40 percent higher accuracy. Examples include determining that a paper would be torn just a little, based how two hands are positioned on the paper in early frames, and predicting that a raised hand, shown facing forward, would swipe down.

"That's important for robotics applications," Zhou says. "You want [a robot] to anticipate and forecast what will happen early on, when you do a specific action."

Next, the researchers aim to improve the module's sophistication. First step is implementing object recognition together with activity recognition. Then, they hope to add in "intuitive physics," meaning helping it understand real-world physical properties of objects. "Because we know a lot of the physics inside these videos, we can train module to learn such physics laws and use those in recognizing new videos," Zhou says. "We also open source all the code and models. Activity understanding is an exciting area of artificial intelligence right now."

                                  https://www.sciencedaily.com/releases/2018/09/180913134545.htm

Record Indoor Magnetic Field

University of Tokyo researchers record highest magnetic field ever achieved indoors -- a discovery that may open doors for materials science and fusion energy research
From: American Institute of Physics

WASHINGTON, D.C., September 17, 2018 -- A group of scientists at the University of Tokyo has recorded the largest magnetic field ever generated indoors -- a whopping 1,200 tesla, as measured in the standard units of magnetic field strength.

By comparison, this is a field strength about 400 times higher than those generated by the huge, powerful magnets used in modern hospital MRI machines, and it is about 50 million times stronger than the Earth's own magnetic field.

Stronger magnetic fields have previously been achieved in outdoor experiments using chemical explosives, but this is a world record for magnetic fields generated indoors in a controlled manner. That greater control means the discovery could open new frontiers in solid-state physics, perhaps allowing scientists to reach what is known as the "quantum limit," a condition where all the electrons in a material are confined to the lowest ground state, where exotic quantum phenomena may appear.

The high magnetic field also has implications for nuclear fusion reactors, a tantalizing if unrealized potential future source of abundant clean energy. To reach the quantum limit or sustain nuclear fusion, scientists believe magnetic field strengths of 1,000 tesla or more may be needed.

The experiments that set the new world record are described in an article appearing this week in the journal Review of Scientific Instruments, from AIP Publishing.

The work opens up a new scientific horizon, said Daisuke Nakamura, first author on the paper, and "has pushed the envelope for ultrahigh magnetic fields."

                        https://www.eurekalert.org/pub_releases/2018-09/aiop-nwr091718.php

Better Infant Disease Detection

By Bill Hathaway, Yale News

September 12, 2018 -- The combination of a new sequencing technique and machine learning can speed up the diagnosis of diseases in newborns and reduce false-positive results, Yale researchers and their collaborators report.

Blood is routinely drawn from a pinprick of an infant’s heel shortly after birth and analyzed for a host of preventable diseases, including more than 40 rare but potentially disabling and serious metabolic disorders. The blood tests are highly sensitive and present a drawback — in many cases they indicate a disorder when none is present.

For instance, reviews of a database of California newborns show that for 100 infants correctly identified as testing positive for methylmalonic acidemia (MMA), an inborn metabolic disorder that causes failure to thrive and can lead to fatal neonatal disease, more than 500 babies were falsely identified as having the disorder.

These false-positive results can cause great anxiety for parents and prompt a battery of tests before they are revealed, said Curt Scharfe, associate professor of genetics and senior author of the new research, which was published Sept. 12 in the journal Genetics in Medicine.

“The time until confirmation is stressful for families, places a burden on the health care system, and in some cases could delay the right treatment for these infants.” Scharfe said.

By introducing a completely new data analysis method, researchers are able to examine an entire metabolic profile that allows for more precise analysis than previous methods, which focus on a smaller fraction of data collected. The new sequencing method developed by the researchers correctly identified 89% of the newborns with MMA.

The team believes it can further increase the sensitivity of MMA gene sequencing to that of another newborn DNA test (for cystic fibrosis) they have clinically validated recently. However, “by combining sequencing with our novel machine learning, we’ve made a big difference in reducing MMA false-positive results,” Scharfe said.

The new combination of sequencing and data analysis can be used to complement existing routine blood work to avoid lengthy testing and speed up the treatment for babies in need of early and additional care, the authors say. Following further clinical validation, the goal is to implement the new testing methods by working with public Newborn Screening (NBS) programs in the U.S. and worldwide.

The Yale team includes the lead authors Gang Peng, a postdoctoral fellow at the Departments of Genetics and Biostatistics at Yale; Neeru Gandotra, associate research scientist in genetics; and Hongyu Zhao, professor of biostatistics, of genetics, and of statistics and data science.

Investigators at Stanford University and the University of California-San Francisco also contributed to the paper.

https://news.yale.edu/2018/09/12/new-approach-improves-detection-diseases-birth

The Aral Sea Is Dying

The Aral Sea was an endorheic lake (one with no outflow) lying between Kazakhstan (Aktobe and Kyzylorda Regions) in the north and Uzbekistan (Karakalpakstan autonomous region) in the south. The name roughly translates as "Sea of Islands", referring to over 1,100 islands that had dotted its waters; in the Turkic languages aral means "island, archipelago". The Aral Sea drainage basin encompasses Uzbekistan and parts of Tajikistan, Turkmenistan, Kyrgyzstan, Kazakhstan, Afghanistan and Iran.  

                                                                                           

The Aral Sea in 1989 (left) and 2014 (right)

Formerly one of the four largest lakes in the world with an area of 68,000 km² (26,300 sq mi), the Aral Sea has been shrinking since the 1960s after the rivers that fed it were diverted by Soviet irrigation projects. By 1997, it had declined to 10% of its original size, splitting into four lakes – the North Aral Sea, the eastern and western basins of the once far larger South Aral Sea, and one smaller intermediate lake. By 2009, the southeastern lake had disappeared and the southwestern lake had retreated to a thin strip at the western edge of the former southern sea; in subsequent years, occasional water flows have led to the southeastern lake sometimes being replenished to a small degree. Satellite images taken by NASA in August 2014 revealed that for the first time in modern history the eastern basin of the Aral Sea had completely dried up. The eastern basin is now called the Aralkum Desert.

In an ongoing effort in Kazakhstan to save and replenish the North Aral Sea, a dam project was completed in 2005; in 2008, the water level in this lake had risen by 12 m (39 ft) compared to 2003. Salinity has dropped, and fish are again found in sufficient numbers for some fishing to be viable. The maximum depth of the North Aral Sea is 42 m (138 ft) (as of 2008.)

The shrinking of the Aral Sea has been called "one of the planet's worst environmental disasters". The region's once-prosperous fishing industry has been essentially destroyed, bringing unemployment and economic hardship. The water from the diverted Syr Darya river is used to irrigate about 5 million acres (2 million hectares) of farmland in the Ferghana Valley. The Aral Sea region is also heavily polluted, with consequential serious public health problems.

UNESCO added the historical documents concerning the development of the Aral Sea to its Memory of the World Register as a unique resource to study this "environmental tragedy".

                                             https://en.wikipedia.org/wiki/Aral_Sea

Battle of Britain Day

Battle of Britain Day is the name given to the day of the large-scale aerial battle that took place on 15 September 1940, during the Battle of Britain.

On this day the Luftwaffe embarked on an all-out attack against London. Around 1,500 aircraft took part in the air battles which lasted until dusk. The action was the climax of the Battle of Britain.

In the aftermath of the raid, Hitler postponed Operation Sea Lion [the German plan to invade the island of Great Britain in an amphibious assault]. Having been defeated in daylight, the Luftwaffe turned its attention to The Blitz night campaign which lasted until May 1941.

Battle of Britain Day is now an annual commemoration of the battle in the United Kingdom. In Canada, the commemoration takes place on the third Sunday of September.

Evaluation of the Day’s Events

In the two main engagements, the fighter losses had been about equal. The big difference was the bombers losses. Fighter Command had had greater success against the afternoon attack rather than the morning assault, which it out numbered 2:1. The ratio of German fighters to bombers had been 3:1 in the morning but 5:1 in the afternoon, so there were more targets. The more bombers Kesselring sent, the more were lost.

Kesselring was back where he started. Park's handling of the actions was a masterpiece of aggressive defence, yet he was not under the same pressure as he had been in the battles during August when air battles were so confusing they were hard to control. A big set-piece offensive played into his hands.

Leigh-Mallory claimed the Big Wings had destroyed entire formations of enemy aircraft upon seeing them. He even claimed the RAF outnumbered their opponents in several engagements. In the afternoon battle, he claimed that the Wing could not get into position to break up the bombers in time and was intercepted by German fighters. Still, while that was true he also claimed his units had shot down 105 enemy aircraft and probably destroyed a further 40. He claimed another 18 damaged for the loss of 14 and six pilots.

The claims were a massive exaggeration. But while the Big Wing had proven physically ineffective for the most part, its biggest contribution to the day had been its use as a psychological weapon.  German aircrews had been told the RAF was a defeated force and the German bomber units that had seen the Big Wing form up were quite shocked, and those crews in the badly hit units, including KG 2 and KG 3, that had witnessed head-on collisions with German aircraft were badly shaken.

A German victory on the Battle of Britain Day was unlikely. It could only have been possible if Park had made crass errors and had been caught on the ground. Stephen Bungay postulates that had the loss rates been reversed, Dowding could have replaced those with reinforcements from his C units and carried on. Moreover, during both major engagements Fighter Command had used less than ½ of its strength. It would have been able to meet the Luftwaffe again on the morrow.

Hermann Göring met his staff at Karinhall the following day for a conference. Their assessments of the air battle verged on pure fantasy. They concluded the RAF had withdrawn all available fighter units from all over the country to concentrate on London. The fact that the Western afternoon raids against ports were uncontested led them to believe the enemy was breaking. Another four or five days, they thought, would be sure to break them.

The OKL believed that the British were down to their last 300 fighters, with only 250 being produced. To stop fighter production, factories in Bristol were to be attacked. London was also to be subjected to round the clock bombing.

Theo Osterkamp pointed to the massed formations used by the British (Big Wing), and put their use down to the ineffectiveness of the 15 September raids. Göring was delighted with the news that the British were committing mass formations to battle. The Big Wing's use would allow the German fighters to engage and destroy RAF fighters in large numbers. However, the Germans had not realised this stratagem was by no means the uniform strategy in Fighter Command. Nevertheless, the OKL were still confident of victory. It blamed the bad weather and RAF's last-minute change of strategy for prolonging summer operations. Still, the Luftwaffe bomber crews were ordered to prepare winter quarters. The campaign turned to London for the duration of 1940. It would witness some large-scale daylight air battles, but it gradually turned to a campaign by night. It would become known as The Blitz.

Aftermath: Overclaiming and Propaganda

Overclaiming in aerial warfare is not uncommon. During the Second World War, pilots often claimed more aircraft shot down than was actually the case. The Air Ministry released a press statement on 15 September that 175–185 German aircraft had been shot down. The actual number of aircraft destroyed was ⅔ lower and significantly less than the number of German losses on 15 and 18 August (The Hardest Day), in which the Luftwaffe lost 75 and 69 respectively.

At 20:00, Churchill, who had returned to 10 Downing Street, was awoken. He received bad news from the navy. In the Atlantic sinking of shipping had been bad, but his Secretary informed him that all had been redeemed in the day's air battle. He was told the RAF had downed 183 enemy aircraft for under 40 losses.

On 16 September, a British flying boat arrived in New York City delivering news of a "record bag" of 185 enemy aircraft. The German Embassy tried in vain to correct the total. The Germans were ignored and The New York Times ran several excited stories calling for a military alliance with Britain and her Commonwealth. The Germans were slower in putting their story together. On 17 September, the Nazi Party newspaper Völkischer Beobachter announced that attacks had caused considerable damage to London. It claimed the Luftwaffe destroyed 79 RAF aircraft for 43 losses. This was also a severe over claim. RAF losses amounted to 29 fighters.

AOC 11 Group Keith Park was livid with the claim returns. As far as he was concerned, claiming 200 on one day was nonsense. He placed particular blame on Leigh-Mallory's Big Wing which had claimed one third, around 60, of the 185 'victories' (total claims were 81 in the morning and 104 in the afternoon). More damage should have been done to German bombers inbound to the target rather than destroying stragglers that were no military threat as they made their way to the coast. He complained that there were too many inexperienced leaders and interceptions were being missed. Things needed "tightening up", that was Park's lesson for 15 September. Park was aware the ratio of losses was 2:1 in the British favour. It had been a decent performance, but not Fighter Command's best.

                                             https://en.wikipedia.org/wiki/Battle_of_Britain_Day

Bankruptcy of Lehman Brothers

The filing for Chapter 11 bankruptcy protection by financial services firm Lehman Brothers on September 15, 2008, remains the largest bankruptcy filing in U.S. history, with Lehman holding over US$600,000,000,000 in assets.

The bank had become so deeply involved in mortgage origination that it had effectively become a real estate hedge fund disguised as an investment bank. At the height of the subprime mortgage crisis, it was exceptionally vulnerable to any downturn in real estate values.

The bankruptcy triggered a one-day drop in the Dow Jones Industrial Average of 4.5%, the largest decline since the September 11, 2001 attacks.

Background

Rise of mortgage origination (1997–2006)

Lehman was one of the first Wall Street firms to move into the business of mortgage origination. In 1997, Lehman bought Colorado-based lender Aurora Loan Services, an Alt-A lender. In 2000, to expand their mortgage origination pipeline, Lehman purchased West Coast subprime mortgage lender BNC Mortgage LLC. Lehman quickly became a force in the subprime market. By 2003 Lehman made $18.2 billion in loans and ranked third in lending. By 2004, this number topped $40 billion. By 2006, Aurora and BNC were lending almost $50 billion per month.

Lehman had morphed into a real estate hedge fund disguised as an investment bank. By 2008, Lehman had assets of $680 billion supported by only $22.5 billion of firm capital. From an equity position, its risky commercial real estate holdings were thirty times greater than capital. In such a highly leveraged structure, a three- to five-percent decline in real estate values would wipe out all capital.

Exposure to the mortgage market

Lehman borrowed significant amounts to fund its investing in the years leading to its bankruptcy in 2008, a process known as leveraging or gearing. A significant portion of this investment was in housing-related assets, making it vulnerable to a downturn in that market. One measure of this risk-taking was its leverage ratio, a measure of the ratio of assets to owners equity, which increased from approximately 24:1 in 2003 to 31:1 by 2007. While generating tremendous profits during the boom, this vulnerable position meant that just a 3–4% decline in the value of its assets would entirely eliminate its book value of equity. Investment banks such as Lehman were not subject to the same regulations applied to depository banks to restrict their risk-taking.

In August 2007, Lehman closed its subprime lender, BNC Mortgage, eliminating 1,200 positions in 23 locations, and took a $25-million after-tax charge and a $27-million reduction in goodwill. The firm said that poor market conditions in the mortgage space "necessitated a substantial reduction in its resources and capacity in the subprime space".

Lehman's final months

In 2008, Lehman faced an unprecedented loss due to the continuing subprime mortgage crisis. Lehman's loss resulted from having held onto large positions in subprime and other lower-rated mortgage tranches when securitizing the underlying mortgages. Whether Lehman did this because it was simply unable to sell the lower-rated bonds, or made a conscious decision to hold them, is unclear. In any event, huge losses accrued in lower-rated mortgage-backed securities throughout 2008. In the second fiscal quarter, Lehman reported losses of $2.8 billion and decided to raise $6 billion in additional capital by offering new shares. In the first half of 2008 alone, Lehman stock lost 73% of its value as the credit market continued to tighten. In August 2008, Lehman reported that it intended to release 6% of its work force, 1,500 people, just ahead of its third-quarter-reporting deadline in September.

On August 22, 2008, shares in Lehman closed up 5% (16% for the week) on reports that the state-controlled Korea Development Bank was considering buying Lehman. Most of those gains were quickly eroded as news emerged that Korea Development Bank was "facing difficulties pleasing regulators and attracting partners for the deal." It culminated on September 9, 2008, when Lehman's shares plunged 45% to $7.79, after it was reported that the state-run South Korean firm had put talks on hold.

Investor confidence continued to erode as Lehman's stock lost roughly half its value and pushed the S&P 500 down 3.4% on September 9, 2008. The Dow Jones lost nearly 300 points the same day, on investors' concerns about the security of the bank. The U.S. government did not announce any plans to assist with any possible financial crisis that emerged at Lehman.

On September 10, 2008, Lehman announced a loss of $3.9 billion and their intent to sell off a majority stake in their investment-management business, which included Neuberger Berman. The stock slid 7% that day.

On September 12, 2008, Timothy F. Geithner, then president of the Federal Reserve Bank of New York, called a meeting on the future of Lehman, which included the possibility of an emergency liquidation of its assets. Bankers representing all the major Wall Street firms were in attendance. The meeting goal was to find a private solution in rescuing Lehman and extinguish the flame of the global financial crisis. Lehman reported that it had been in talks with Bank of America and Barclays for the company's possible sale. The New York Times reported on September 14, 2008, that Barclays had ended its bid to purchase all or part of Lehman and a deal to rescue the bank from liquidation collapsed. It emerged subsequently that a deal had been vetoed by the Bank of England and the UK's Financial Services Authority. Leaders of major Wall Street banks continued to meet late that day to prevent the bank's rapid failure. Bank of America's rumored involvement also appeared to end as federal regulators resisted its request for government involvement in Lehman's sale. By Sunday, after the Barclays deal fell through, the news of pending doom swept through Lehman, and many employees arrived at the headquarters to clean out their offices. By Sunday afternoon, the government summoned Harvey Miller of Weil, Gotshal & Manges to file for bankruptcy before the markets opened on Monday.

https://en.wikipedia.org/wiki/Bankruptcy_of_Lehman_Brothers

RNA Test Indicates Viral Infection

By Ziba Kashef, Yale News

December 17, 2017 -- A new test that measures RNA or protein molecules in human cells can accurately identify viral infection as a cause of respiratory symptoms, according to a Yale study published Dec. 21 in The Journal of Infectious Diseases. Performed with a simple nasal swab, the test could prove to be a quicker, cheaper way to diagnose respiratory viral illnesses than current methods, the researchers said.

“It’s a simpler test and more cost-effective for looking at viral infection,” said author Ellen Foxman, M.D., assistant professor of laboratory medicine at Yale School of Medicine.

Upper respiratory illnesses are common, yet there is no rapid diagnostic test to confirm more than a handful of common viruses as the cause. To identify biomarkers, or indicators, of viral infection applicable to many different respiratory viruses, Foxman and co-author Marie Landry, M.D., first tested human nasal cells in the laboratory. With genetic sequencing techniques, they screened the cells for RNAs and proteins that increase when a virus is present.

Foxman and Landry identified three RNAs, and two proteins, that are “turned on” by a virus. They then investigated whether measuring the expression of the genes, or levels of the proteins, could predict the presence of a viral infection.

The researchers found that the RNAs and proteins were both accurate predictors of respiratory viral infection, confirmed by subsequent testing for common viruses. The RNAs predicted viral infection with 97% accuracy. This method also picked up viruses that are not identified by many current lab tests, they said.

“Instead of looking for individual viruses, our test asks the question: ‘Is the body fighting a virus?’” said Foxman. “We found we can answer that question very well.”

The researchers hope to develop the method into a rapid gene or protein test that doctors could perform in their offices. Such a test could help providers diagnose a viral infection more quickly and accurately than with routine evaluation or more time-consuming and expensive tests, the researchers said.

The test could be particularly useful for assessing very sick patients or young children, they added, and it could also help reduce the misuse of antibiotics to treat viral infections.

“One reason to test is to know why the patient is sick,” said Foxman. “The other reason is to make a decision about whether people who are not that sick should get antibiotics.”

The research team’s goal is to create a gene- or protein-based test available for general use within one to five years, Foxman said.

This work was supported by the National Institutes of Health and Biomedical Advanced Research and Development Authority, and by the Yale Department of Laboratory Medicine. Foxman and Landry have a patent pending.

https://news.yale.edu/2017/12/21/new-test-shows-when-body-fighting-virus