Cloud Databases

• Learn an introduction to Cloud Databases
• Learn about the types of Cloud Databases
• Learn when to use the right type of Cloud Database
• Learn how to manage IBM Cloud Databases
• Exercise Create a Cloudant NoSQL Database
• Exercise Integrate Database to Spring Boot Application
• Exercise Extend Spring Boot app with CRUD methods

Introduction

• Databases For PostgreSQL — A SQL database for persisting relational data that conforms to a schema that can be queryed using SQL
• Cloudant — A NoSQL database for persisting schemaless data structured as JSON documents
• Cloud Object Storage — For persisting binary data, especially huge sets of data

• MySQL: MySQL is an open-source relational database management system (RDBMS) based on Structured Query Language (SQL). MySQL runs on virtually all platforms, including Linux, UNIX, and Windows.
• PostgreSQL: PostgreSQL, often simply Postgres, is an object-relational database management system (ORDBMS) with an emphasis on extensibility and standards compliance.
• DB2: DB2 is an RDBMS designed to store, analyze, and retrieve data efficiently.

• Relational databases are well-documented and mature technologies, and RDBMSs are sold and maintained by a number of established corporations.
• SQL standards are well-defined and commonly accepted.
• All RDBMS are ACID-compliant, meaning their transactions are Atomic, Consistent, Isolated, and Durable

• RDBMSs don’t work well — or at all — with unstructured or semi-structured data due to schema and type constraints. This makes them ill-suited for large analytics or IoT event loads.
• The tables in your relational database will not necessarily map one-to-one with an object or class representing the same data.
• When migrating one RDBMS to another, schemas and types must generally be identical between source and destination tables for migration to work (schema constraint). For many of the same reasons, extremely complex datasets or those containing variable-length records are generally difficult to handle with an RDBMS schema.

• Key-value stores, such as Redis , are extremely simple database management systems that store only key-value pairs and provide basic functionality for retrieving the value associated with a known key. The simplicity of key-value stores makes these database management systems particularly well-suited to embedded databases, where the stored data is not particularly complex and speed is of paramount importance.

• Wide column stores, such as Cassandra and Scylla are schema-agnostic systems that enable users to store data in column families or tables, a single row of which can be thought of as a record — a multi-dimensional key-value store. These solutions are designed with the goal of scaling well enough to manage petabytes of data across as many as thousands of commodity servers in a massive, distributed system. Although technically schema-free, wide column stores like Scylla and Cassandra use an SQL variant called CQL for data definition and manipulation, making them straightforward to those already familiar with RDBMS.

• Document stores, including MongoDB and Cloudant, are schema-free systems that store data in the form of JSON documents. Document stores are similar to key-value or wide column stores, but the document name is the key and the contents of the document, whatever they are, are the value. In a document store, individual records do not require a uniform structure, can contain many different value types, and can be nested. This flexibility makes them particularly well-suited to manage semi-structured data across distributed systems.

• Graph databases, such as Neo4J and Datastax Enterprise Graph, represent data as a network of related nodes or objects in order to facilitate data visualizations and graph analytics. A node or object in a graph database contains free-form data that is connected by relationships and grouped according to labels. Graph-oriented database management systems (DBMS) software is designed with an emphasis on illustrating connections between data points. As a result, graph databases are typically used when analysis of the relationships between heterogeneous data points is the end goal of the system, such as in fraud prevention, advanced enterprise operations, or Facebook’s original friend’s graph.

• Search engines, such as Elasticsearch and Solr, store data using schema-free JSON documents. They are similar to document stores, but with a greater emphasis on making your unstructured or semi-structured data easily accessible via text-based searches with strings of varying complexity.

• Schema-free data models are more flexible and easier to administer. NoSQL databases are generally more horizontally scalable and fault-tolerant.
• Data can easily be distributed across different nodes. To improve availability and/or partition tolerance, you can choose that data on some nodes be eventually consistent.

• NoSQL databases are generally less widely adopted and mature than RDBMS solutions, so specific expertise is often required.
• There is a range of formats and constraints specific to each database type.

• If your first priority is ACID transactions (Atomic, Consistent, Isolated, and Durable), consider using RDBMS.
• If you have a massively distributed system and can settle for eventual consistency on some nodes/partitions, you might consider a wide column store such as Cassandra or Scylla.
• If your input data is particularly heterogeneous and difficult to encapsulate according to a normalization schema, consider using a NoSQL DBMS.
• If your goal is to scale vertically, consider an RDBMS; conversely, if you want to scale horizontally, a NoSQL DBMS may be preferable. \ = Managing IBM Cloud Databases

Creating a Database

• Create the Database instance
• Populate it with sample data

• While logged into the IBM Cloud, click on the Catalog link in the header

• Select the Cloudant NoSQL DB
  • You should see the Database creation screen,
  • Change the name to cloudant-<machine name>
  • Select Use Both for available authentication methods.
  • Select the resource group you are working in.
  • Click Create button to create the database instance.

• This will create a database instance that you can start working with to store your product information.

• After the create is successful, you will see the Database instance Manage view. From here you can click on the Launch button to access the database management views. We now need to configure the command line so you can upload data.

• Click on the Service Credentials on the left-hand menu.

• If there are no credentials created, click New Credentials button and in the dialog click Add. These control the access to the database. Are the credentials have been created you should see a screen similar to the one below.

• .Open a command prompt and create a folder/directory called data

mkdir data

• https://next.json-generator.com

[
  {
    'repeat(1, 100)': {
      _id: '{{objectId()}}',
      index: '{{index()}}',
      name: '{{lorem(2, "words")}}',
      guid: '{{guid()}}',
      delivered: '{{bool()}}',
      price: '{{floating(1000, 4000, 2, "$0,0.00")}}',
Copy to clipboardCopied!

• Copy the generated contents on the right hand side into a file called products-<machine name>.json and save it into the same folder. Wrap the array with a docs statement.

{
  "docs": <Add Generated array []>
}
Copy to clipboardCopied!

• Using a code editor create a shell script called load.sh that will load data into the database using a curl command. Use the contents from below for the script.

#!/bin/bash
# load.sh
UIDPW=
AUTH="$(python -c 'import base64; print base64.urlsafe_b64encode("<<username>>:<<password>>")')"
ACURL="curl -s --proto '=https' -iv -g -H 'Authorization: Basic ${AUTH}'"
HOST="https://<<host>>"
# Products Data
eval ${ACURL} -X DELETE '${HOST}/<<machine name>>'
Copy to clipboardCopied!

• Replace the username:password fields with values from the Credentials section of the Cloudant instance in the dashboard. . Replace the host with the value from the credentials section of the dashboard. Replace machine name with name of your machine to create a unique database name. .Save the script, make it executable, and then run it.

chmod +x ./load.sh
./load.sh
Copy to clipboardCopied!

• The data from the products-<machine name>.json file will then be used to populate the database, to confirm this on the Dashboard click on Manage menu on the left and then Launch button to see the Cloudant dashboard.

• Click on the Left icon that looks like a Database and you will see the products database created.

• Click on the products database itself.

• You can see the rows of data

• If you click on a row of data, you will see the raw NoSQL form of the data record.

• This completes the setup of the database and populating it with data. We will be now moving onto creating a Spring Micro service that then uses this database to display Product details.

Consuming in a Spring micro service

• Create a Products Micro service using Spring backend Starter Kit
• Make some updates to integrate the app programmatically with the database
• Test these changes locally

• image:createapp.png[]

OpenAPI json : http://localhost:8080/swagger/api

cloudant.username=<Username>
cloudant.password=<Password>
cloudant.url=<URL>
cloudant.db=<machine name>
Copy to clipboardCopied!

<dependency>
    <groupId>com.cloudant</groupId>
    <artifactId>cloudant-client</artifactId>
    <version>2.11.0</version>
</dependency>
Copy to clipboardCopied!

• Now we need to add the code that will read the configuration and then bookstrap the Cloudant SDK.

• Create a file in src/main/java/application called CloudantConfiguration.java and add the following code.

package application;
import com.cloudant.client.api.ClientBuilder;
import com.cloudant.client.api.CloudantClient;
import com.cloudant.client.api.Database;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
Copy to clipboardCopied!

package application;
import java.net.URL;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix="cloudant")
public class CloudantConfigurationProperties {
    private URL url;
Copy to clipboardCopied!

package application.model;
import java.util.Objects;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonCreator;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import org.springframework.validation.annotation.Validated;
import javax.validation.Valid;
Copy to clipboardCopied!

• Using the Starter Kit it helps as it has already defined a Products API using Spring Boot. We can now extend this with the Cloudant API calls to access the contents of the database.

• Open src/main/java/application/ProductsApiBinding.java, add the following import.

import com.cloudant.client.api.Database;
import java.io.IOException;
Copy to clipboardCopied!

@Autowired
private Database db;
----
. Final step is to add the code to `getAll` this retrieves the stored documents in the database. Replace the contents of the `getAll` method with the following code.
Copy to clipboardCopied!

ApiResponseMessage ApiRm = null;
try {
    List<Product> allDocs = db.getAllDocsRequestBuilder().includeDocs(true).build().getResponse().getDocsAs(Product.class);
    ApiRm = new ApiResponseMessage<>(HttpStatus.FOUND, "", allDocs);
} catch (IOException e) {
    ApiRm = new ApiResponseMessage<>(HttpStatus.NOT_FOUND, "", null);
}
Copy to clipboardCopied!

. You can see the code using the `db` instance to call Cloudant Couch DB APIs and return a List of documents, this is then wrapped into a `ApiResponseMessage` so a consistent handling of HTTP response messages can be achieved. Save any changes and rerun the micro service.
. Open the Swagger UI endpoint listed in the console logs. You should see a Swagger API looking similar to the screen shot below.
+
![Swagger](/images/swaggerapi.png)
. Expand the *GET* `/products` API endpoint and select *Try It* The response body should now display an array of all the documents stored in the NoSQL database.
+
Copy to clipboardCopied!

. Using a NoSQL database allows for lots of flexibility for the document format and they have a lot of rich APIs.
. Congratulations ! In this short exercise, you have created and deployed a Java Spring microservice and integrated to an IBM Cloud Cloudant database instance.
### References
-  https://console.bluemix.net/docs/services/Cloudant/getting-started.html#getting-started-with-cloudant
- http://spring.io/projects/spring-boot
- https://cloud.ibm.com/
Copy to clipboardCopied!