Wednesday, December 12, 2018

GCP: Cloud Pub/sub

Cloud Pub/Sub is a simple, reliable, scalable foundation for stream analytics and event-driven computing systems
As part of Google Cloud’s stream analytics solution, the service ingests event streams and delivers them to Cloud Dataflow for processing and BigQuery for analysis as a data warehousing solution.  Relying on the Cloud Pub/Sub service for delivery of event data frees one to focus on transforming their business and data systems with applications such as:

Real-time personalization in gaming
Fast reporting, targeting and optimization in advertising and media
Processing device data for healthcare, manufacturing, oil and gas, and logistics
Syndicating market-related data streams for financial services

Build multi cloud and hybrid applications on open architecture.
Syndicate data across projects and applications running on other clouds, or between cloud and on-premises apps. Cloud Pub/Sub easily fits in your existing environment via efficient client libraries for multiple languages, open REST/HTTP and gRPC service APIs, and an open source Apache Kafka connector.

Scale responsively and automatically
Scale to hundreds of millions of messages per second and pay only for the resources you use. There are no partitions or local instances to manage, reducing operational overhead. Data is automatically and intelligently distributed across data centers over our unique, high-speed private network.

Bring reliability and security tools to real-time apps
Use Cloud Pub/Sub to simplify scalable, distributed systems. All published data is synchronously replicated across availability zones to ensure that messages are available to consumers for processing as soon as they are ready. Fine-grained access controls allow for sophisticated cross-team and organizational data sharing. And end-to-end encryption adds security to your pipelines.

Below is how Cloud Pub/Sub can be used

- Applications, Devices, Databases can ingest data into Cloud Pub/Sub
- Which then get processed in the Cloud data flow.
- For Analysis, they get fed into Cloud Big Query, Cloud machine learning, Cloud Big Table
- cloud big query uses Data Studio, Third party tools for data warehousing.

Cloud machine learning can be used for predictive learning, Cloud big table is used for Caching and serving.

references:
https://cloud.google.com/pubsub

1 comment: