If a system of machine is generating data with 5GB per day, and it is analyzed in real-time using spark. How can we combine it with the batch historian data that is stored every day. In case if the result is stored on daily bases, weekly or monthly basis. I don't have applied and having no experience to do like that. So plz can some one discuss the overall frame work or know some links or papers. I read something about Lambda Architecture but did not find any practical framework with step wise guidance. Thanks
Dear Maqbool,
I suggest you to see links, attached files and publications in topics.
How to Use Data Streaming for Big Data - dummies
www.dummies.com/...data/data.../how-to-use-data-streaming-for-b...
datascience-ai-machinelearning-resources/Big Data, Analytics, and ...
https://github.com/.../datascience.../Big%20Data%2C%20Analytics...
Streaming Data: Big Data at High Velocity - Upwork
https://www.upwork.com › Hiring Headquarters Home › Data
Ingestion and Processing of Data for Big Data and IoT Solutions
https://www.xenonstack.com/.../ingestion-processing-data-for-big-...
Best regards
Conference Paper Faster, larger, easier: reining real-time big data processing in cloud
Article A task-level adaptive MapReduce framework for real-time stre...
Conference Paper A column store engine for real-time streaming analytics
Conference Paper Parallel Job Processing Technique for Real-time Big-Data Pro...
Article IJESRT INTERNATIONAL JOURNAL OF ENGINEERING SCIENCES & RESEA...
Article Squall: A Real-time Big Data Processing Framework based on T...
Thesis A model of programming languages for dynamic real-time strea...
- Badges
- Science topic