- Table View
- List View
Data-Driven Services with Silverlight 2: Data Access and Web Services for Rich Internet Applications
by John PapaThis comprehensive book teaches you how to build data-rich business applications with Silverlight 2 that draw on multiple sources of data. Packed with reusable examples, Data-Driven Services with Silverlight 2 covers all of the data access and web service tools you need, including data binding, the LINQ data querying component, RESTful and SOAP web service support, cross-domain web service calls, and Microsoft's new ADO.NET Data Services and the ADO.NET Entity Framework. With this book, you will: Know when and how to use LINQ to JSON, LINQ to XML, and LINQ to Objects Learn how Silverlight 2 applications bind, pass, read, save, query, and present data Discover how your application can call web services to work with SOAP, REST, RSS, AtomPub, POX and JSON Design REST, ASMX, and WCF web services that communicate with Silverlight 2 Harness RESTful web services such as Digg, Amazon, and Twitter Retrieve and save data using the new Entity Framework and WCF Work with RESTful ADO.NET Data Services and its Silverlight client library to move data between your Silverlight application and a database Data-Driven Services with Silverlight 2 offers many tips and tricks for building data-rich business applications, and covers the scenarios you're most likely to encounter. Complete examples in C# and VB can be downloaded from the book's companion website.
Data Driven Smart Manufacturing Technologies and Applications (Springer Series in Advanced Manufacturing)
by Weidong Li Yuchen Liang Sheng WangThis book reports innovative deep learning and big data analytics technologies for smart manufacturing applications. In this book, theoretical foundations, as well as the state-of-the-art and practical implementations for the relevant technologies, are covered. This book details the relevant applied research conducted by the authors in some important manufacturing applications, including intelligent prognosis on manufacturing processes, sustainable manufacturing and human-robot cooperation. Industrial case studies included in this book illustrate the design details of the algorithms and methodologies for the applications, in a bid to provide useful references to readers. Smart manufacturing aims to take advantage of advanced information and artificial intelligent technologies to enable flexibility in physical manufacturing processes to address increasingly dynamic markets. In recent years, the development of innovative deep learning and big data analytics algorithms is dramatic. Meanwhile, the algorithms and technologies have been widely applied to facilitate various manufacturing applications. It is essential to make a timely update on this subject considering its importance and rapid progress. This book offers a valuable resource for researchers in the smart manufacturing communities, as well as practicing engineers and decision makers in industry and all those interested in smart manufacturing and Industry 4.0.
Data-Driven Storytelling (AK Peters Visualization Series)
by Nathalie Henry Riche Christophe Hurter Nicholas Diakopoulos Sheelagh CarpendaleThis book presents an accessible introduction to data-driven storytelling. Resulting from unique discussions between data visualization researchers and data journalists, it offers an integrated definition of the topic, presents vivid examples and patterns for data storytelling, and calls out key challenges and new opportunities for researchers and practitioners.
Data Driven Strategies: Theory and Applications
by Wang Jianhong Ricardo A. Ramirez-Mendoza Ruben Morales-MenendezA key challenge in science and engineering is to provide a quantitative description of the systems under investigation, leveraging the noisy data collected. Such a description may be a complete mathematical model or a mechanism to return controllers corresponding to new, unseen inputs. Recent advances in the theories are described in detail, along with their applications in engineering. The book aims to develop model-free system analysis and control strategies, i.e., data-driven control from theoretical analysis and engineering applications based only on measured data. The study aims to develop system identification, and combination in advanced control theory, i.e., data-driven control strategy as system and controller are generated from measured data directly. The book reviews the development of system identification and its combination in advanced control theory, i.e., data-driven control strategy, as they all depend on measured data. Firstly, data-driven identification is developed for the closed-loop, nonlinear system and model validation, i.e., obtaining model descriptions from measured data. Secondly, the data-driven idea is combined with some control strategies to be considered data-driven control strategies, such as data-driven model predictive control, data-driven iterative tuning control, and data-driven subspace predictive control. Thirdly data-driven identification and data-driven control strategies are applied to interested engineering. In this context, the book provides algorithms to perform state estimation of dynamical systems from noisy data and some convex optimization algorithms through identification and control problems.
Data-Driven Systems and Intelligent Applications (Intelligent Data-Driven Systems and Artificial Intelligence)
by Mangesh M. Ghonge N. Krishna Chaitanya Pradeep N Harish Garg Alessandro BrunoThis book comprehensively discusses basic data-driven intelligent systems, the methods for processing the data, and cloud computing with artificial intelligence. It presents fundamental and advanced techniques used for handling large user data, and for the data stored in the cloud. It further covers data-driven decision-making for smart logistics and manufacturing systems, network security, and privacy issues in cloud computing.This book: Discusses intelligent systems and cloud computing with the help of artificial intelligence and machine learning. Showcases the importance of machine learning and deep learning in data-driven and cloud-based applications to improve their capabilities and intelligence. Presents the latest developments in data-driven and cloud applications with respect to their design and architecture. Covers artificial intelligence methods along with their experimental result analysis through data processing tools. Presents the advent of machine learning, deep learning, and reinforcement technique for cloud computing to provide cost-effective and efficient services. The text will be useful for senior undergraduate, graduate students, and academic researchers in diverse fields including electrical engineering, electronics and communications engineering, computer engineering, manufacturing engineering, and production engineering.
Data-Driven Technologies and Artificial Intelligence in Supply Chain: Tools and Techniques (Intelligent Data-Driven Systems and Artificial Intelligence)
by Mahesh Chand Vineet Jain Puneeta AjmeraThis book highlights the importance of data-driven technologies and artificial intelligence in supply chain management. It covers important concepts such as enabling technologies in Industry 4.0, the impact of artificial intelligence, and data-driven technologies in lean manufacturing. "Provides solutions to solve complex supply chain management issues using artificial intelligence and data-driven technologies" Emphasizes the impact of a data-driven supply chain on quality management "Discusses applications of artificial intelligence, and data-driven technologies in the service industry, and lean manufacturing" Highlights the barriers to implementing artificial intelligence in small and medium enterprises Presents a better understanding of different risks such as procurement risks, process risks, demand risks, transportation risks, and operational risks The book comprehensively discusses the applications of artificial intelligence and data-driven technologies in supply chain management for diverse fields such as service industries, manufacturing industries, and healthcare. It further covers the impact of artificial intelligence and data-driven technologies in managing the FMGC supply chain. It will be a valuable resource for senior undergraduate, graduate students, and academic researchers in diverse fields including electrical engineering, electronics and communications engineering, industrial engineering, manufacturing engineering, production engineering, and computer engineering.
Data Economy in the Digital Age (Data-Intensive Research)
by Samiksha Shukla Kritica Bisht Kapil Tiwari Shahid BashirThe book is a comprehensive guide that explores the concept of data economy and its implications in today's world. The book discusses the principles and components of the ecosystem, the challenges and opportunities presented by data monetization, and the potential risks related to data privacy. Real-life examples and case studies are included to understand the concepts better. The book is suitable for individuals in data science, economics, business, and technology and for students, academics, and policymakers. It is an excellent read for anyone interested in the data economy.
Data-Enabled Analytics: DEA for Big Data (International Series in Operations Research & Management Science #312)
by Joe Zhu Vincent CharlesThis book explores the novel uses and potentials of Data Envelopment Analysis (DEA) under big data. These areas are of widespread interest to researchers and practitioners alike. Considering the vast literature on DEA, one could say that DEA has been and continues to be, a widely used technique both in performance and productivity measurement, having covered a plethora of challenges and debates within the modelling framework.
Data Engineering: Mining, Information and Intelligence (International Series in Operations Research & Management Science #132)
by John Talburt Terry M. Talley Yupo ChanDATA ENGINEERING: Mining, Information, and Intelligence describes applied research aimed at the task of collecting data and distilling useful information from that data. Most of the work presented emanates from research completed through collaborations between Acxiom Corporation and its academic research partners under the aegis of the Acxiom Laboratory for Applied Research (ALAR). Chapters are roughly ordered to follow the logical sequence of the transformation of data from raw input data streams to refined information. Four discrete sections cover Data Integration and Information Quality; Grid Computing; Data Mining; and Visualization. Additionally, there are exercises at the end of each chapter. The primary audience for this book is the broad base of anyone interested in data engineering, whether from academia, market research firms, or business-intelligence companies. The volume is ideally suited for researchers, practitioners, and postgraduate students alike. With its focus on problems arising from industry rather than a basic research perspective, combined with its intelligent organization, extensive references, and subject and author indices, it can serve the academic, research, and industrial audiences.
Data Engineering and Applications: Proceedings of the International Conference, IDEA 2K22, Volume 2 (Lecture Notes in Electrical Engineering #1189)
by Jitendra Agrawal Rajesh K. Shukla Sanjeev Sharma Chin-Shiuh ShiehThis book comprises select proceedings from the 4th International Conference on Data, Engineering, and Applications (IDEA 2022). The contents discuss novel contributions and latest developments in the domains of data structures and data management algorithms, information retrieval and information integration, social data analytics, IoT and data intelligence, Industry 4.0 and digital manufacturing, data fusion, natural language processing, geolocation handling, image, video and signal processing, ICT applications and e-governance, among others. This book is of interest to researchers in academia and industry working in big data, data mining, machine learning, data science, and their associated learning systems and applications.
Data Engineering and Applications: Proceedings of the International Conference, IDEA 2K22, Volume 1 (Lecture Notes in Electrical Engineering #1146)
by Jitendra Agrawal Rajesh K. Shukla Sanjeev Sharma Chin-Shiuh ShiehThis book comprises select proceedings from the 4th International Conference on Data, Engineering, and Applications (IDEA 2022). The contents discuss novel contributions and latest developments in the domains of data structures and data management algorithms, information retrieval and information integration, social data analytics, IoT and data intelligence, Industry 4.0 and digital manufacturing, data fusion, natural language processing, geolocation handling, image, video and signal processing, ICT applications and e-governance, among others. This book is of interest to researchers in academia and industry working in big data, data mining, machine learning, data science, and their associated learning systems and applications.
Data, Engineering and Applications: Select Proceedings of IDEA 2021 (Lecture Notes in Electrical Engineering #907)
by Sanjeev Sharma Sheng-Lung Peng Jitendra Agrawal Rajesh K. Shukla Dac-Nhuong LeThe book contains select proceedings of the 3rd International Conference on Data, Engineering, and Applications (IDEA 2021). It includes papers from experts in industry and academia that address state-of-the-art research in the areas of big data, data mining, machine learning, data science, and their associated learning systems and applications. This book will be a valuable reference guide for all graduate students, researchers, and scientists interested in exploring the potential of big data applications.
Data, Engineering and Applications: Volume 2
by Rajesh Kumar Shukla Jitendra Agrawal Sanjeev Sharma Geetam Singh TomerThis book presents a compilation of current trends, technologies, and challenges in connection with Big Data. Many fields of science and engineering are data-driven, or generate huge amounts of data that are ripe for the picking. There are now more sources of data than ever before, and more means of capturing data. At the same time, the sheer volume and complexity of the data have sparked new developments, where many Big Data problems require new solutions. Given its scope, the book offers a valuable reference guide for all graduate students, researchers, and scientists interested in exploring the potential of Big Data applications.
Data Engineering and Communication Technology: Proceedings of ICDECT 2020 (Lecture Notes on Data Engineering and Communications Technologies #63)
by K. Ashoka Reddy B. Rama Devi Boby George K. Srujan RajuThis book includes selected papers presented at the 4th International Conference on Data Engineering and Communication Technology (ICDECT 2020), held at Kakatiya Institute of Technology & Science, Warangal, India, during 25–26 September 2020. It features advanced, multidisciplinary research towards the design of smart computing, information systems and electronic systems. It also focuses on various innovation paradigms in system knowledge, intelligence and sustainability which can be applied to provide viable solutions to diverse problems related to society, the environment and industry.
Data Engineering and Intelligent Computing: Proceedings of 5th ICICC 2021, Volume 1 (Lecture Notes in Networks and Systems #446)
by Vikrant Bhateja Lai Khin Wee Jerry Chun-Wei Lin Suresh Chandra Satapathy T. M. RajeshThis book features a collection of high-quality, peer-reviewed papers presented at the Fifth International Conference on Intelligent Computing and Communication (ICICC 2021) organized by the Department of Computer Science and Engineering and the Department of Computer Science and Technology, Dayananda Sagar University, Bengaluru, India, on 26–27 November 2021. The book is organized in two volumes and discusses advanced and multi-disciplinary research regarding the design of smart computing and informatics. It focuses on innovation paradigms in system knowledge, intelligence and sustainability that can be applied to provide practical solutions to a number of problems in society, the environment and industry. Further, the book also addresses the deployment of emerging computational and knowledge transfer approaches, optimizing solutions in various disciplines of science, technology and health care.
Data Engineering and Intelligent Computing: Proceedings of IC3T 2016 (Advances in Intelligent Systems and Computing #542)
by Suresh Chandra Satapathy Vikrant Bhateja K. Srujan Raju B. JanakiramaiahThe book is a compilation of high-quality scientific papers presented at the 3rd International Conference on Computer & Communication Technologies (IC3T 2016). The individual papers address cutting-edge technologies and applications of soft computing, artificial intelligence and communication. In addition, a variety of further topics are discussed, which include data mining, machine intelligence, fuzzy computing, sensor networks, signal and image processing, human-computer interaction, web intelligence, etc. As such, it offers readers a valuable and unique resource.
Data Engineering and Intelligent Computing: Proceedings of ICICC 2020 (Advances in Intelligent Systems and Computing #1)
by Suresh Chandra Satapathy Vikrant Bhateja V. N. Manjunath Aradhya Carlos M. Travieso-GonzálezThis book features a collection of high-quality, peer-reviewed papers presented at the Fourth International Conference on Intelligent Computing and Communication (ICICC 2020) organized by the Department of Computer Science and Engineering and the Department of Computer Science and Technology, Dayananda Sagar University, Bengaluru, India, on 18–20 September 2020. The book is organized in two volumes and discusses advanced and multi-disciplinary research regarding the design of smart computing and informatics. It focuses on innovation paradigms in system knowledge, intelligence and sustainability that can be applied to provide practical solutions to a number of problems in society, the environment and industry. Further, the book also addresses the deployment of emerging computational and knowledge transfer approaches, optimizing solutions in various disciplines of science, technology and health care.
Data Engineering for Machine Learning Pipelines: From Python Libraries to ML Pipelines and Cloud Platforms
by Pavan Kumar NarayananThis book covers modern data engineering functions and important Python libraries, to help you develop state-of-the-art ML pipelines and integration code. The book begins by explaining data analytics and transformation, delving into the Pandas library, its capabilities, and nuances. It then explores emerging libraries such as Polars and CuDF, providing insights into GPU-based computing and cutting-edge data manipulation techniques. The text discusses the importance of data validation in engineering processes, introducing tools such as Great Expectations and Pandera to ensure data quality and reliability. The book delves into API design and development, with a specific focus on leveraging the power of FastAPI. It covers authentication, authorization, and real-world applications, enabling you to construct efficient and secure APIs using FastAPI. Also explored is concurrency in data engineering, examining Dask's capabilities from basic setup to crafting advanced machine learning pipelines. The book includes development and delivery of data engineering pipelines using leading cloud platforms such as AWS, Google Cloud, and Microsoft Azure. The concluding chapters concentrate on real-time and streaming data engineering pipelines, emphasizing Apache Kafka and workflow orchestration in data engineering. Workflow tools such as Airflow and Prefect are introduced to seamlessly manage and automate complex data workflows. What sets this book apart is its blend of theoretical knowledge and practical application, a structured path from basic to advanced concepts, and insights into using state-of-the-art tools. With this book, you gain access to cutting-edge techniques and insights that are reshaping the industry. This book is not just an educational tool. It is a career catalyst, and an investment in your future as a data engineering expert, poised to meet the challenges of today's data-driven world. What You Will Learn Elevate your data wrangling jobs by utilizing the power of both CPU and GPU computing, and learn to process data using Pandas 2.0, Polars, and CuDF at unprecedented speeds Design data validation pipelines, construct efficient data service APIs, develop real-time streaming pipelines and master the art of workflow orchestration to streamline your engineering projects Leverage concurrent programming to develop machine learning pipelines and get hands-on experience in development and deployment of machine learning pipelines across AWS, GCP, and Azure Who This Book Is For Data analysts, data engineers, data scientists, machine learning engineers, and MLOps specialists
Data Engineering for Smart Systems: Proceedings of SSIC 2021 (Lecture Notes in Networks and Systems #238)
by Priyadarsi Nanda Vivek Kumar Verma Sumit Srivastava Rohit Kumar Gupta Arka Prokash MazumdarThis book features original papers from the 3rd International Conference on Smart IoT Systems: Innovations and Computing (SSIC 2021), organized by Manipal University, Jaipur, India, during January 22–23, 2021. It discusses scientific works related to data engineering in the context of computational collective intelligence consisted of interaction between smart devices for smart environments and interactions. Thanks to the high-quality content and the broad range of topics covered, the book appeals to researchers pursuing advanced studies.
Data Engineering in Medical Imaging: Second MICCAI Workshop, DEMI 2024, Held in Conjunction with MICCAI 2024, Marrakesh, Morocco, October 10, 2024, Proceedings (Lecture Notes in Computer Science #15265)
by Binod Bhattarai Sharib Ali Anita Rau Razvan Caramalau Anh Nguyen Prashnna Gyawali Ana Namburete Danail StoyanovThis book constitutes the proceedings of the Second MICCAI Workshop on Data Engineering in Medical Imaging, DEMI 2024, held in conjunction with the 27th International conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2024, in Marrakesh, Morocco, on October 10, 2024. The 18 papers presented in this book were carefully reviewed and selected. These papers focus on the application of various Data engineering techniques in the field of Medical Imaging.
Data Engineering in Medical Imaging: First MICCAI Workshop, DEMI 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023, Proceedings (Lecture Notes in Computer Science #14314)
by Binod Bhattarai Sharib Ali Anita Rau Anh Nguyen Ana Namburete Razvan Caramalau Danail StoyanovVolume LNCS 14414 constitutes the refereed proceedings of the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, which was held in Vancouver, Canada in October 2023.The DEMI 2023 proceedings contain 11 high-quality papers of 9 to 15 pages pre-selected through a rigorous peer review process (with an average of three reviews per paper). All submissions were peer-reviewed through a double-blind process by at least three members of the scientific review committee, comprising 16 experts in the field of medical imaging. The accepted manuscripts cover various medical image analysis methods and applications.
Data Engineering on Azure
by Vlad RiscutiaBuild a data platform to the industry-leading standards set by Microsoft&’s own infrastructure.Summary In Data Engineering on Azure you will learn how to: Pick the right Azure services for different data scenarios Manage data inventory Implement production quality data modeling, analytics, and machine learning workloads Handle data governance Using DevOps to increase reliability Ingesting, storing, and distributing data Apply best practices for compliance and access control Data Engineering on Azure reveals the data management patterns and techniques that support Microsoft&’s own massive data infrastructure. Author Vlad Riscutia, a data engineer at Microsoft, teaches you to bring an engineering rigor to your data platform and ensure that your data prototypes function just as well under the pressures of production. You'll implement common data modeling patterns, stand up cloud-native data platforms on Azure, and get to grips with DevOps for both analytics and machine learning. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology Build secure, stable data platforms that can scale to loads of any size. When a project moves from the lab into production, you need confidence that it can stand up to real-world challenges. This book teaches you to design and implement cloud-based data infrastructure that you can easily monitor, scale, and modify. About the book In Data Engineering on Azure you&’ll learn the skills you need to build and maintain big data platforms in massive enterprises. This invaluable guide includes clear, practical guidance for setting up infrastructure, orchestration, workloads, and governance. As you go, you&’ll set up efficient machine learning pipelines, and then master time-saving automation and DevOps solutions. The Azure-based examples are easy to reproduce on other cloud platforms. What's inside Data inventory and data governance Assure data quality, compliance, and distribution Build automated pipelines to increase reliability Ingest, store, and distribute data Production-quality data modeling, analytics, and machine learning About the reader For data engineers familiar with cloud computing and DevOps. About the author Vlad Riscutia is a software architect at Microsoft. Table of Contents 1 Introduction PART 1 INFRASTRUCTURE 2 Storage 3 DevOps 4 Orchestration PART 2 WORKLOADS 5 Processing 6 Analytics 7 Machine learning PART 3 GOVERNANCE 8 Metadata 9 Data quality 10 Compliance 11 Distributing data
Data Engineering with Alteryx: Helping data engineers apply DataOps practices with Alteryx
by Paul HoughtonBuild and deploy data pipelines with Alteryx by applying practical DataOps principlesKey FeaturesLearn DataOps principles to build data pipelines with AlteryxBuild robust data pipelines with Alteryx DesignerUse Alteryx Server and Alteryx Connect to share and deploy your data pipelinesBook DescriptionAlteryx is a GUI-based development platform for data analytic applications.Data Engineering with Alteryx will help you leverage Alteryx's code-free aspects which increase development speed while still enabling you to make the most of the code-based skills you have.This book will teach you the principles of DataOps and how they can be used with the Alteryx software stack. You'll build data pipelines with Alteryx Designer and incorporate the error handling and data validation needed for reliable datasets. Next, you'll take the data pipeline from raw data, transform it into a robust dataset, and publish it to Alteryx Server following a continuous integration process.By the end of this Alteryx book, you'll be able to build systems for validating datasets, monitoring workflow performance, managing access, and promoting the use of your data sources.What you will learnBuild a working pipeline to integrate an external data sourceDevelop monitoring processes for the pipeline exampleUnderstand and apply DataOps principles to an Alteryx data pipelineGain skills for data engineering with the Alteryx software stackWork with spatial analytics and machine learning techniques in an Alteryx workflow Explore Alteryx workflow deployment strategies using metadata validation and continuous integrationOrganize content on Alteryx Server and secure user accessWho this book is forIf you're a data engineer, data scientist, or data analyst who wants to set up a reliable process for developing data pipelines using Alteryx, this book is for you. You'll also find this book useful if you are trying to make the development and deployment of datasets more robust by following the DataOps principles. Familiarity with Alteryx products will be helpful but is not necessary.
Data Engineering with Apache Spark, Delta Lake, and Lakehouse: Create scalable pipelines that ingest, curate, and aggregate complex data in a timely and secure way
by Manoj Kukreja Danil ZburivskyUnderstand the complexities of modern-day data engineering platforms and explore strategies to deal with them with the help of use case scenarios led by an industry expert in big dataKey FeaturesBecome well-versed with the core concepts of Apache Spark and Delta Lake for building data platformsLearn how to ingest, process, and analyze data that can be later used for training machine learning modelsUnderstand how to operationalize data models in production using curated dataBook DescriptionIn the world of ever-changing data and schemas, it is important to build data pipelines that can auto-adjust to changes. This book will help you build scalable data platforms that managers, data scientists, and data analysts can rely on.Starting with an introduction to data engineering, along with its key concepts and architectures, this book will show you how to use Microsoft Azure Cloud services effectively for data engineering. You'll cover data lake design patterns and the different stages through which the data needs to flow in a typical data lake. Once you've explored the main features of Delta Lake to build data lakes with fast performance and governance in mind, you'll advance to implementing the lambda architecture using Delta Lake. Packed with practical examples and code snippets, this book takes you through real-world examples based on production scenarios faced by the author in his 10 years of experience working with big data. Finally, you'll cover data lake deployment strategies that play an important role in provisioning the cloud resources and deploying the data pipelines in a repeatable and continuous way.By the end of this data engineering book, you'll know how to effectively deal with ever-changing data and create scalable data pipelines to streamline data science, ML, and artificial intelligence (AI) tasks.What you will learnDiscover the challenges you may face in the data engineering worldAdd ACID transactions to Apache Spark using Delta LakeUnderstand effective design strategies to build enterprise-grade data lakesExplore architectural and design patterns for building efficient data ingestion pipelinesOrchestrate a data pipeline for preprocessing data using Apache Spark and Delta Lake APIsAutomate deployment and monitoring of data pipelines in productionGet to grips with securing, monitoring, and managing data pipelines models efficientlyWho this book is forThis book is for aspiring data engineers and data analysts who are new to the world of data engineering and are looking for a practical guide to building scalable data platforms. If you already work with PySpark and want to use Delta Lake for data engineering, you'll find this book useful. Basic knowledge of Python, Spark, and SQL is expected.
Data Engineering with AWS: Learn how to design and build cloud-based data transformation pipelines using AWS
by Gareth Eagar Rafael Pecora Marcos AmorimThe missing expert-led manual for the AWS ecosystem — go from foundations to building data engineering pipelines effortlesslyPurchase of the print or Kindle book includes a free eBook in the PDF format.Key FeaturesLearn about common data architectures and modern approaches to generating value from big dataExplore AWS tools for ingesting, transforming, and consuming data, and for orchestrating pipelinesLearn how to architect and implement data lakes and data lakehouses for big data analytics from a data lakes expertBook DescriptionWritten by a Senior Data Architect with over twenty-five years of experience in the business, Data Engineering for AWS is a book whose sole aim is to make you proficient in using the AWS ecosystem. Using a thorough and hands-on approach to data, this book will give aspiring and new data engineers a solid theoretical and practical foundation to succeed with AWS. As you progress, you'll be taken through the services and the skills you need to architect and implement data pipelines on AWS. You'll begin by reviewing important data engineering concepts and some of the core AWS services that form a part of the data engineer's toolkit. You'll then architect a data pipeline, review raw data sources, transform the data, and learn how the transformed data is used by various data consumers. You'll also learn about populating data marts and data warehouses along with how a data lakehouse fits into the picture. Later, you'll be introduced to AWS tools for analyzing data, including those for ad-hoc SQL queries and creating visualizations. In the final chapters, you'll understand how the power of machine learning and artificial intelligence can be used to draw new insights from data. By the end of this AWS book, you'll be able to carry out data engineering tasks and implement a data pipeline on AWS independently.What you will learnUnderstand data engineering concepts and emerging technologiesIngest streaming data with Amazon Kinesis Data FirehoseOptimize, denormalize, and join datasets with AWS Glue StudioUse Amazon S3 events to trigger a Lambda process to transform a fileRun complex SQL queries on data lake data using Amazon AthenaLoad data into a Redshift data warehouse and run queriesCreate a visualization of your data using Amazon QuickSightExtract sentiment data from a dataset using Amazon ComprehendWho this book is forThis book is for data engineers, data analysts, and data architects who are new to AWS and looking to extend their skills to the AWS cloud. Anyone new to data engineering who wants to learn about the foundational concepts while gaining practical experience with common data engineering services on AWS will also find this book useful.A basic understanding of big data-related topics and Python coding will help you get the most out of this book but it's not a prerequisite. Familiarity with the AWS console and core services will also help you follow along.