A new machine learning tool for efficient processing of large amounts of data

(ORDO NEWS) — Large amounts of data have become a major challenge for astronaut scientists analyzing them.

To address this issue, a team of researchers developed a machine learning tool to efficiently label large, complex datasets to allow deep learning models to filter out and identify potentially hazardous solar events. The new labeling tool can be applied or adapted for other tasks related to huge datasets.

“Labeling data with multivalued annotations is a critical step in supervised machine learning (ML). However, labeling datasets is tedious and time-consuming,” says Dr. Subhamoy Chatterjee, of SwRI, who specializes in solar astronomy and instrumentation.

“New research shows how Convolutional Neural Networks (CNNs) trained on roughly labeled astronomical videos can be used to improve the quality and breadth of data labeling and reduce the need for human intervention.”

Deep learning methods automate the processing and interpretation of large amounts of complex data by extracting and learning complex patterns.

The SwRI team used video footage of the solar magnetic field to identify regions where strong, complex magnetic fields occur on the solar surface, which are a major predictor of space weather events.

“When we trained, we used coarse labels, manually checking only our discrepancies with the machine,” said study co-author Dr. Andrés Muñoz-Jaramillo, a solar energy physicist at SwRI who specializes in machine learning.

We then retuned the algorithm with the corrected data and repeated this process until we reached a consensus.” While labeling a new stream is usually done manually, this iterative interaction between a human and a machine learning algorithm reduces manual verification by 50%.”

Iterative labeling approaches such as active learning can save significant time by reducing the cost of preparing big data for ML.

Moreover, by gradually labeling the video and looking for the moment when the MO algorithm changes its classification, the SwRI scientists further used the customized MO algorithm to create a richer and more useful database.

“We created an end-to-end deep learning approach to classify videos of magnetic spot evolution without explicitly providing segmented images, tracking algorithms, or other hand-crafted features,” said Dr. Derek Lamb of SwRI, co-author of the study who specializes in the evolution of magnetic fields on the surface of the Sun.

“This database will be critical to developing new methodologies for predicting the occurrence of complex regions that contribute to space weather events, potentially increasing the time we have to prepare for space weather.”


Contact us: [email protected]

Our Standards, Terms of Use: Standard Terms And Conditions.