IndicGenBench Google Research India recently released IndicGenBench, a multilingual benchmark to evaluate generation capabilities of LLMs on 29 Indic languages spanning 13 writing scripts and 4 language families. Extended the datasets in Cross-lingual Summarization, Machine Translation, Multi-lingual Question Answering, and Cross-lingual Question Answering, the team has collected human translations for English examples into target Indic languages, thereby extending the scope and applicability of evaluation metrics in this domain. One of the key insights from their study is the analysis of token fertility across all Indic languages within IndicGenBench. Token fertility, representing the average number of sub-words that a word is broken down into by the tokenizer, varies significantly across languages Some languages have simple breakdowns, while others are more complex. Now, why does this matter? Well, it affects how well the language models work. Languages with more complex breakdowns might struggle because they can't use as many examples to learn from. They found that languages with simpler breakdowns can use more examples effectively compared to those with complex breakdowns. #LLMs #GenAI #IndicGenBench #AI #IndicLanguages #IndicDatasets #multilingual

15

1 Comment

🔥 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈 𝐚𝐧𝐝 𝐋𝐋𝐌 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 - 𝐓𝐨𝐩 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 🔥 🤖 I've conducted a mock interview with my student, Bheesetti Anand, focused on Generative AI and LLM (Large Language Models) interview questions. This session is designed to benefit other students preparing for similar interviews, helping them secure their dream roles in the field of Generative AI. 📔 Here’s why watching the video can help you make a 𝐬𝐮𝐜𝐜𝐞𝐬𝐬𝐟𝐮𝐥 𝐜𝐚𝐫𝐞𝐞𝐫 𝐬𝐰𝐢𝐭𝐜𝐡 𝐭𝐨 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈: ✅ 𝐂𝐨𝐦𝐩𝐫𝐞𝐡𝐞𝐧𝐬𝐢𝐯𝐞 𝐂𝐨𝐯𝐞𝐫𝐚𝐠𝐞: Detailed exploration of top interview questions related to Generative AI and Large Language Models. ✅ 𝐏𝐫𝐚𝐜𝐭𝐢𝐜𝐚𝐥 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬: Real-world examples and practical applications to understand complex concepts better. ✅ 𝐄𝐱𝐩𝐞𝐫𝐭 𝐆𝐮𝐢𝐝𝐚𝐧𝐜𝐞: Tips and strategies to approach and answer interview questions effectively. ✅ 𝐂𝐚𝐫𝐞𝐞𝐫 𝐏𝐫𝐞𝐩𝐚𝐫𝐚𝐭𝐢𝐨𝐧: Essential knowledge and skills to prepare for interviews and stand out to potential employers. 📽 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰 𝐥𝐢𝐧𝐤: https://lnkd.in/g_nkP7fs #generativeai #llm #interview #jobpreparation #mockinterview

14

2 Comments

Jianbo Shi and Jitendra Malik's paper on "Normalized Cuts and Image Segmentation" (https://lnkd.in/dcyU5Skr) published in 2000, is a seminal work that introduces an interesting method for image segmentation by using concepts from graph theory and linear algebra. It is one of the most highly cited papers in Computer Vision with over 20k+ citations. By modeling the image as a graph, where pixels represent nodes and the edges between them encode the similarity, the segmentation problem transforms into one of graph partitioning. The segments then can be thought of as subgraphs. The key idea is to partition the graph to minimize the normalized cut (a criterion that evaluates the cut cost: the total weight of the edges removed to split the graph) relative to the total edge connections within each group. This problem is then elegantly converted into a linear algebra problem: finding the partition corresponds to solving for the second smallest eigenvector of the Laplacian matrix of the graph. Here's a short report on the entire paper, prepared as part of our project component for the Linear Algebra and its Applications course last semester. Please feel free to comment or address any issues if there are any! #ImageSegmentation #MachineLearning #ComputerVision #DataScience #ScientificResearch #LinearAlgebra #GraphTheory #Report #project

125

5 Comments

It’s been an honor working with Gil Weinberg! Thanks so much! This demo is from one of the chapters in my thesis on #AI driven raga agnostic automatic accompaniment for alapana (non-rhythmic improvisations) in South Indian Classical Music. Since the algorithm is raga agnostic, the system can accompany for any raga performed! More demos on a violin playing #robot coming soon! #MusicTechnology #MachineLearning #HumanRobotInteraction #HumanComputerInteraction #AutomaticAccompaniment

217

31 Comments

EMNLP 2024 results are out, we got in three papers, congrats to students :) First paper of many, Vatsal, Pranshu, Suyash, Janit, Kunal, Srija and Adnan, Congrats 🎉 🎊 🍾 🎈 Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets (main) - Vatsal Gupta, PRANSHU PANDYA, Tushar Kataria, Vivek Gupta, Dan Roth tldr: make models robust on multiple adversarial challenge set simultaneously  arxiv: https://lnkd.in/ei-65Y6g Knowledge-Aware Reasoning over Multimodal Semi-structured Tables (finding) Suyash Vardhan Mathur, Jainit Bafna, Kunal Kartik, Harshita Khandelwal, Manish Shrivastava, Vivek Gupta, Mohit Bansal, Dan Roth tldr: introduce the novel task of reasoning on multimodal tabular data, text + images in a structure (first of a kind) arxiv: https://lnkd.in/e5ycusfn   Unraveling the Truth: Do LLMs really Understand Charts? A Deep Dive into Consistency and Robustness (finding) Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, Dan Roth           tldr: VLM although seem very accurate on chart, are not all robust and consistent in performance arxiv: https://lnkd.in/eFFA-sVX We also got an workshop paper, at BlockBoxNLP 2024 @EMNLP, here the details Enhancing Question Answering on Charts Through Effective Pre-training Tasks Ashim Gupta, Vivek Gupta, Shuo Zhang, Yujie He, Ning Zhang , Shalin Shah, Ph.D. tldr: models is not robust & consistent on charts, can we use pre-training on simple auxiliary tasks to enhance model robustness arxiv: https://lnkd.in/egbRHm-4

385

12 Comments

🚨ECCV 2024 Paper Alert 🚨 ➡️Paper Title: T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning 🌟Few pointers from the paper 🎯The scarcity of annotated data in LiDAR point cloud understanding hinders effective representation learning. Consequently, scholars have been actively investigating efficacious self-supervised pre-training paradigms. 🎯Nevertheless, temporal information, which is inherent in the LiDAR point cloud sequence, is consistently disregarded. 🎯To better utilize this property, authors of this paper proposed an effective pre-training strategy, namely “Temporal Masked Auto-Encoders (T-MAE)”, which takes as input temporally adjacent frames and learns temporal dependency. 🎯A SiamWCA backbone, containing a Siamese encoder and a windowed cross-attention (WCA) module, is established for the two-frame input. 🎯Considering that the movement of an ego-vehicle alters the view of the same instance, temporal modeling also serves as a robust and natural data augmentation, enhancing the comprehension of target objects. SiamWCA is a powerful architecture but heavily relies on annotated data. 🎯Their T-MAE pre-training strategy alleviates its demand for annotated data. Comprehensive experiments demonstrate that T-MAE achieves the best performance on both Waymo and ONCE datasets among competitive self-supervised approaches. 🏢Organization:  University of Amsterdam 🧙Paper Authors: Weijie Wei, Fatemeh Karimi Nejadasl, PhD, Theo Gevers, Martin R. Oswald 1️⃣Read the Full Paper here: https://lnkd.in/gKMuGJM5 2️⃣Project Page: https://lnkd.in/g5CC6a7H 3️⃣Code: https://lnkd.in/gHXXt6Gf 🎥 Be sure to watch the attached Presentation - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements. #ECCV2024 #lidarpointcloud #selfsupervisedlearning

24

Project specific call for internship: We are looking for 2 summer interns in India in summer 2025 for the Quantum Algorithm and Engineering (QAE) team. The goal of these internships will be to understand domain specific best practices (e.g. error mitigation, qiskit-addons, transpilation) to execute utility-scale projects on IBM Quantum hardware using Qiskit, and bring this knowledge to the community in the form of one or more tutorial notebooks. We are specifically looking for students who have a strong understanding of either (i) quantum for bioscience, or (ii) parallel and serverless computing for quantum middleware. Good grasp of Qiskit 1.0 or higher is necessary. Also note that an intern must be a student and should not graduate in 2025. Please apply for IBM Quantum summer internship in India here: https://lnkd.in/gPfaaxFg If you are interested in any of these two projects, please send me a message over linkedin after applying (or if you applied previously via this link). Also note that (i) sending me a message does not guarantee an internship position, but it will help me during screening, and (ii) I am not aware of other quantum projects for summer 2025, so please don't ask for that. Nate Earnest-Noble #ibm #ibmquantum #quantumcomputing #qiskit

207

26 Comments

Brain Teaser #10: Suppose we are fine-tuning a large language model using Low-Rank Adaptation (LoRA). We possess two low-rank update matrices, δWA​ and δWB​​, both associated with the rank "r". Considering our objective to preserve the model's current knowledge while incorporating new data from diverse features, what "mathematical rationale" should guide the selection of one of the update matrices (δWA​ or δWB) for low-rank adaptation? Hint: Consider the specific characteristics or properties of the matrices that can help in developing mathematical reasoning. Here is my response: The norm ratio, calculated as the ratio of the nuclear norm to the Frobenius norm of the update matrix, is an essential metric for assessing how updates impact a model. It provides insights into the model's stability and adaptability. Here’s a brief overview: https://lnkd.in/ekB_q9XX #MachineLearning #DeepLearning #NaturalLanguageProcessing #LoRA #FineTuning #AI #DataScience #ModelOptimization #NeuralNetworks #ArtificialIntelligence #TechInnovation #DataDriven #KnowledgeIntegration #Research #LLMs

30

4 Comments

Excited to share our latest paper titled - 𝐇𝐨𝐰 𝐭𝐨 𝐂𝐡𝐨𝐨𝐬𝐞 𝐚 𝐓𝐡𝐫𝐞𝐬𝐡𝐨𝐥𝐝 𝐟𝐨𝐫 𝐚𝐧 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐌𝐞𝐭𝐫𝐢𝐜 𝐟𝐨𝐫 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬. This work addresses the critical gap in research on setting robust thresholds for LLM evaluation metrics, which are essential for the reliable deployment of LLMs. Drawing inspiration from model risk management practices in regulated industries like finance, we propose a systematic methodology to define these thresholds. The approach involves identifying risks associated with the specific application, understanding stakeholders' risk tolerance, and applying statistically rigorous procedures using ground-truth data.

59

4 Comments

🔎 Demystifying LLMs: A New Video Series! 🔎 Struggling to wrap your head around Large Language Models (LLMs)? I feel you! That's why I'm launching a new video series, LLM Explained, in collaboration with Uthiralakshmi S. from Worcester Polytechnic Institute. Each episode will be a bite-sized 2-minute explainer, tackling a specific LLM concept. This week, we're diving into the foundations: tokenization and embedding! Whether you're a seasoned data scientist or just starting your AI journey, this series is designed to make LLMs accessible and engaging.✨ Stay tuned for the next episode! In the meantime, hit that follow button and join the conversation! If you learned something valuable, share it with your network! 👨‍💻 👥 #GenAI #LLM #LargeLanguaeModels #DeepLearning #NeuralNetwork #AI #MachineLearning #Tokens #Tokenization #Embedding #Education #AIEducation #Data #DataScience #NEU #WPI

18

2 Comments

Fine tuning gpt2 on indic-gpt dataset I ran one epoch of fine tuning on an arbitrary dataset of Indic-gpt and managed to train it for language detection with Colab.Have a look: https://lnkd.in/dZJUY-UV #indic-gpt #fine tuning #language detection

10

Recommendation systems have become crucial in helping users navigate vast amounts of content. You might have been recommended to my post because of LinkedIn recommender system. Traditionally, recommendation systems have relied on techniques such as collaborative filtering and content-based filtering. These methods have served us well, but the emergence of Large Language Models (LLMs) has opened up a new frontier of possibilities in this field. I learned a lot about RecSys and Information Retrieval from Sumit Kumar's newsletter. My Experiment I've created a simple yet effective LLM-based recommendation system using: -Netflix dataset from Kaggle -Local LLaMA2 model running on Ollama -FAISS for similarity search In my approach, I use an LLM to generate dense vector representations (embeddings) of movie descriptions. These embeddings capture semantic meaning, allowing us to find similar movies based on their content. Then I use FAISS, a library for efficient similarity search, to quickly find the most similar movies to a given title. To generate embeddings for each title. I am using a local API (LLaMA2 running locally on Ollama) to generate embeddings. Check out my detailed Medium post https://lnkd.in/edsXF2b5 Dataset: https://lnkd.in/eFtmRST4 Code : https://lnkd.in/eS4wd5TV What's your take on using LLMs for recommendations? Have you experimented with this approach? What's your approach for generating embeddings? #AI #MachineLearning #RecommendationSystems #LLMs

268

11 Comments

Predictions based on Machine Learning and Statistics often rely on maximal likelihood, which can trap LLMs in biases inherited from the majority opinions of training data. This problem has been substantially addressed through conditional statistics, allowing for mitigation of biases and preventing the cancellation of minority perspectives. https://lnkd.in/gMXKqwyB

7

Is GPT-4o good at table recognition? Not quite! OCR looks fine, but tripped by spanning cell table structure! 🤩 However, our UniTable, with only 124M param trained by self-supervision, shows great understanding! 👉 UniTable paper: arxiv.org/abs/2403.04822 👉 UniTable code: https://lnkd.in/e4a7YMxJ #MachineLearning #GPT-4o #Table #Document

85

8 Comments

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

51

2 Comments

🚀 Excited to present our work on "AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations" at the #NeurIPS2024 Workshop on Adaptive Foundation Models! 📅 December 14, 2024, 10:45 AM 📍 West Exhibition Hall A 🔴 Problem: Multimodal web agents struggle to adapt to unseen tasks/websites/domains, relying on large-scale pre-training and fine-tuning, which limits efficiency. 🛠️ Methods: We introduce AdaptAgent, enabling agents to adapt with few-shot human demonstrations via in-context learning for proprietary models and meta-learning for open-weight models. 📈 Results & Importance: AdaptAgent boosts task success by a relative increase of 21.03% to 65.75% with just 1-2 multimodal demonstrations, offering a more efficient, adaptable solution for web agents. 📄 For more details, check out our arxiv paper: https://lnkd.in/g46s2Pys. This work is led by our intern Gaurav Verma during his time at J.P. Morgan AI Research, along with fellow collaborators Nishan S., Zhen Zeng, Tucker Richard Balch, and Manuela Veloso ❤️🤝 PS: We’re also looking for an intern in multimodal agents! 🤖 If you're passionate about multimodal systems, reach out (here or at NeurIPS 2024) or apply: https://lnkd.in/gtN6P5w9. #MultimodalAgents #FewShotLearning #JobOpening

7

Day 5 of learning in public 🚀 Traditional fine-tuning methods for large language models often face challenges like high computational requirements, catastrophic forgetting, and single-task specialization. Enter LoRA (Low-Rank Adaptation), a game-changing technique that addresses these limitations, enabling efficient and practical fine-tuning of large language models. LoRA introduces a novel approach to fine-tuning by adding a small number of task-specific weights to the pre-trained model instead of updating all the weights. Here's how it works: 1. Low-Rank Matrices: For each layer in the pre-trained model, LoRA introduces a pair of low-rank matrices: one for the weights and one for the biases. 2. Adaptation: During fine-tuning, LoRA updates these low-rank matrices instead of modifying the pre-trained weights directly. 3. Composition: At inference time, the low-rank matrices are composed with the pre-trained weights to produce the final, adapted weights for the fine-tuned model. The low-rank nature of these matrices is the key to LoRA's efficiency, dramatically reducing the memory and computational requirements for fine-tuning. This makes it possible to adapt large language models even on consumer-grade hardware like CPUs or edge devices. But LoRA's advantages don't stop there. It can match or even outperform the performance of full fine-tuning while using far fewer updatable parameters. Additionally, the LoRA updates are reversible, allowing the model to revert to its original state, and composable, enabling a single pre-trained model to be fine-tuned for multiple tasks simultaneously. To illustrate the power of LoRA, let's dive into a practical example. Here, I'll demonstrate how to implement LoRA from scratch and fine-tune the DistilBERT model using LoRA on the IMDB dataset for sentiment analysis. https://lnkd.in/g-Tz2NU7 As you can see, LoRA is a remarkable technique that addresses the limitations of traditional fine-tuning methods, making it more efficient, practical, and versatile. By introducing low-rank, task-specific updates, LoRA enables efficient adaptation while preserving the model's original knowledge and allowing for multi-task specialization. As the demand for deploying large language models on resource-constrained devices continues to grow, techniques like LoRA will make these powerful models more accessible and practical.

4