GSF's method involves the disintegration of the input tensor with grouped spatial gating, followed by the fusion of these components using channel weighting. Transforming 2D CNNs into high-performing spatio-temporal feature extractors is feasible through the addition of GSF, with negligible increases in both parameters and computational cost. Through an in-depth analysis of GSF, employing two prevalent 2D CNN architectures, we obtain state-of-the-art or competitive outcomes on five widely recognized benchmarks for action recognition tasks.
Inferencing with embedded machine learning models at the edge necessitates a careful consideration of the trade-offs between resource metrics like energy and memory usage and performance metrics like processing speed and prediction accuracy. Our research surpasses traditional neural network methods, investigating the Tsetlin Machine (TM), an emerging machine learning algorithm. This approach employs learning automata to formulate propositional logic rules for classification. programmed stimulation A novel methodology for training and inference of TM is developed using algorithm-hardware co-design principles. By utilizing independent training and inference techniques for transition machines, the REDRESS methodology seeks to shrink the memory footprint of the resultant automata, facilitating their use in low-power and ultra-low-power applications. The Tsetlin Automata (TA) array stores binary information, signifying excludes (0) and includes (1), encapsulating the knowledge acquired. By storing only the inclusion data, REDRESS's include-encoding method delivers over 99% compression efficiency for lossless TA compression. HRO761 By employing a novel and computationally minimal training procedure, Tsetlin Automata Re-profiling, the accuracy and sparsity of TAs are improved, decreasing the number of inclusions and, hence, the memory footprint. REDRESS's inference mechanism, based on a fundamentally bit-parallel algorithm, processes the optimized trained TA directly in the compressed domain, avoiding decompression during runtime, and thus achieves considerable speed gains in comparison to the current state-of-the-art Binary Neural Network (BNN) models. This study showcases that the REDRESS method results in superior TM performance compared to BNN models across all design metrics on five benchmark datasets. In the field of machine learning, the datasets MNIST, CIFAR2, KWS6, Fashion-MNIST, and Kuzushiji-MNIST hold importance. REDRESS's performance on the STM32F746G-DISCO microcontroller produced speed and energy gains ranging from 5 to 5700 times compared to the different BNN models.
Deep learning-driven fusion techniques have exhibited promising efficacy in the realm of image fusion. The network architecture's substantial involvement in the fusion process is responsible for this observation. Generally speaking, determining an effective fusion architecture proves difficult; consequently, the engineering of fusion networks remains largely a black art, not a precisely defined scientific method. We mathematically define the fusion task in order to resolve this issue, establishing a correlation between its optimal solution and the corresponding network architecture. The paper proposes a novel, lightweight fusion network construction method stemming from this approach. The method bypasses the time-intensive practice of empirically designing networks by employing a strategy of trial and error. Specifically, we employ a learnable representation method for the fusion process, where the fusion network's architectural design is influenced by the optimization algorithm shaping the learned model. The low-rank representation (LRR) objective forms the basis of our learnable model. The solution's fundamental matrix multiplications are recast as convolutional operations, and the iterative optimization process is superseded by a dedicated feed-forward network. From this pioneering network architecture, an end-to-end, lightweight fusion network is built, aiming to combine infrared and visible light images. Its successful training hinges upon a detail-to-semantic information loss function, meticulously designed to maintain the image details and augment the significant characteristics of the original images. Our experiments demonstrate that the proposed fusion network surpasses the current leading fusion methods in terms of fusion performance, as evaluated on publicly available datasets. Our network, to our surprise, needs fewer training parameters in comparison to other existing methods.
Deep models for visual recognition face a significant hurdle in learning from long-tailed datasets, requiring the training of robust deep architectures on a large number of images following this distribution. Deep learning, a powerful recognition model, has taken center stage in the last ten years, revolutionizing the learning of high-quality image representations and driving remarkable advancements in generic visual recognition. However, the skewed representation of classes, a common difficulty in practical visual recognition, frequently restricts the practicality of deep network-based recognition models in real-world applications, because of their susceptibility to bias toward dominant classes and poor performance on less common ones. In response to this challenge, a substantial volume of research has been undertaken in recent years, yielding encouraging advancements within the field of deep long-tailed learning. This paper undertakes a comprehensive survey on the latest advancements in deep long-tailed learning, acknowledging the rapid development of this field. To be exact, we have separated existing deep long-tailed learning studies into three principal classes: class re-balancing, information augmentation, and module enhancement. We will now explore these approaches in depth, following this classification system. Afterwards, we empirically examine multiple state-of-the-art approaches through evaluation of their treatment of class imbalance, employing a novel metric—relative accuracy. pathologic outcomes In the concluding section of the survey, we spotlight the critical applications of deep long-tailed learning and identify some exciting prospective research directions.
Objects within a single scene possess diverse degrees of interconnectedness, yet only a specific subset of these relationships merits attention. Guided by the Detection Transformer's superior object detection performance, we consider scene graph generation to be a set-predictive operation. We propose Relation Transformer (RelTR), an end-to-end scene graph generation model, built with an encoder-decoder structure within this paper. While the encoder examines the visual feature context, the decoder, through the application of various attention mechanisms, deduces a fixed-size collection of subject-predicate-object triplets, coupling subject and object queries. We create a specialized set prediction loss for end-to-end training, dedicated to aligning the predicted triplets with the corresponding ground truth triplets. RelTR's one-step methodology diverges from other scene graph generation methods by directly predicting sparse scene graphs using only visual cues, eschewing entity aggregation and the annotation of all possible relationships. Experiments across the Visual Genome, Open Images V6, and VRD datasets highlight our model's quick inference and superior performance.
In a multitude of visual applications, the identification and characterization of local features are frequently employed, driven by high industrial and commercial needs. For large-scale applications, these tasks place a premium on both the speed and accuracy of local features. Learning local features in existing studies usually centers around the individual characteristics of keypoints, but the relationships between these points, as established from a broad spatial perspective, are often overlooked. Employing a consistent attention mechanism (CoAM), AWDesc, as presented in this paper, facilitates local descriptor awareness of image-level spatial context, both during training and matching. By using a feature pyramid in combination with local feature detection, more stable and accurate keypoint localization can be achieved. Two forms of AWDesc are presented to address the diverse demands in local feature characterization, balancing accuracy and speed. Employing Context Augmentation, we introduce non-local contextual information into convolutional neural networks to alleviate the inherent locality issue, thereby broadening the scope of local descriptors and improving descriptive power. The Adaptive Global Context Augmented Module (AGCA) and the Diverse Surrounding Context Augmented Module (DSCA) are innovative modules for building robust local descriptors, enriching them with global and surrounding context information. Conversely, a remarkably lightweight backbone network is designed, combined with a novel knowledge distillation strategy, to optimize the balance between accuracy and speed. We meticulously conducted experiments on image matching, homography estimation, visual localization, and 3D reconstruction, revealing that our method surpasses the leading local descriptors in the current state-of-the-art. Within the GitHub repository, located at https//github.com/vignywang/AWDesc, you will find the AWDesc code.
For 3D vision tasks, such as registration and identification, consistent correspondences among point clouds are indispensable. We articulate a mutual voting procedure in this paper, for the purpose of ranking 3D correspondences. Reliable scoring for correspondences within a mutual voting scheme is achievable by optimizing the refinement process of both the voters and the candidates. Initially, a graph is constructed, incorporating the pairwise compatibility constraint, based on the initial correspondence set. Next, nodal clustering coefficients are incorporated to initially remove a subset of outliers, thereby expediting the subsequent voting process. Nodes, as candidates, and edges, as voters, form the basis of our third model. Within the graph, mutual voting is employed to ascertain the score of correspondences. In conclusion, the correspondences are prioritized according to their vote totals, and the top-ranked correspondences are identified as inliers.