In microservices architecture, each service is a separate entity and can be developed, deployed, scaled and maintained independently. This project is a simple example of an Ecommerce microservices project. It consists of the following services:
The choice of communication between services will be RESTful API. The options are:
- RestTemplate (deprecated),
- Feign Client (Declarative HTTP Client: Provides a more concise and easier way to call other services)
- WebClient (Reactive & Recommended: Supports asynchronous, non-blocking calls). Asynchronous and non-blocking calls are calls recommended for microservices architecture.
Asynchronous calls: These allow a program to initiate a task and move on to other tasks before the initial task is completed. This helps in utilizing resources more efficiently and improving the overall performance of the application.
Non-blocking calls: These ensure that the program does not wait for the task to complete before moving on to the next task. This is particularly useful in I/O operations where waiting for a response can be time-consuming.
In the context of microservices, asynchronous and non-blocking calls are recommended because they help in handling multiple service requests concurrently without blocking the execution of other tasks, leading to better scalability and performance
Eureka Server (Service Discovery: Used to register and discover services).
IMPORTANT NOTE: The project will be implemented using the web client. When microservices need to call each other dynamically, they register with the Eureka server and use the Eureka client with for service discovery.
API Gateway: This is a single entry point for all the services. It is a reverse proxy that routes requests to the appropriate service.
Steps:
Add dependencies (cloud-starter-netflix-eureka-client, Starter WebFlux (For WebClient), spring actuators) in the pom.xml file for the following services:
- PRODUCT-SERVICE: provides the product details (product id, name, description, price, etc.) to the inventory service and order service. When new products are added, the product service will notify the inventory service to initialize the stock for the product.
- ORDER-SERVICE: provides the order details (order id, product id, user id, quantity, etc.) and calls the product service to get the product details. It will also call the inventory service to check if the product is available in stock and update the stock after placing the order. It will also call the user service to get the user details.
- POTENTIAL ISSUE: When an order is updated, the original ordered quantity is not returned to the inventory service. The new quantity is simply deducted without considering the previous deduction. This leads to an incorrect stock count/level.
- SOLUTION: Track the original ordered quantity in the order service and return it to the inventory service when updating the stock. This way, the inventory service can correctly adjust the stock level.
-
It will be used to send notifications to the users when an order is placed or updated. -
It will call the user service to get the user details and send the notification. -
It will also call the order service to get the order details and send the notification. -
It will also call the product service to get the product details and send the notification.
-
This is a service that will be used to manage the users of the application. -
It will provide APIs to create, update, delete and get the user details. -
It will also provide APIs to authenticate the users. -
It will call the order service to get the order details and user details.
-
orders can be fulfilled. It will call the product service to get the product details and update the stock. -
It will also call the order service to get the order details and update the stock.
-
It will be used to route the requests to the appropriate service. -
It will also be used to provide a single entry point for all the services.
-
It will also be used to provide security for the services. -
It will also be used to provide rate limiting for the services. -
It will also be used to provide load balancing for the services.
NOTIFICATION-SERVICE:
-
This is a service that will be used to send notifications to the users when an order is placed or updated. -
It will call the user service to get the user details and send the notification. -
It will also call the order service to get the order details and send the notification. -
It will also call the product service to get the product details and send the notification.
PAYMENT-SERVICE:
-
This is a service that will be used to process the payments for the orders. -
It will call the order service to get the order details and process the payment. -
It will also call the user service to get the user details and process the payment.
SHIPPING-SERVICE:
-
This is a service that will be used to manage the shipping of the orders. -
It will call the order service to get the order details and manage the shipping. -
It will also call the user service to get the user details and manage the shipping.
CART-SERVICE:
-
This is a service that will be used to manage the cart of the users. -
It will call the product service to get the product details and manage the cart. -
It will also call the user service to get the user details and manage the cart.
REVIEW/RATING-SERVICE:
-
This is a service that will be used to manage the reviews and ratings of the products. -
It will call the product service to get the product details and manage the reviews and ratings. -
It will also call the user service to get the user details and manage the reviews and ratings.
ANALYSIS/REPORTING-SERVICE:
-
This is a service that will be used to analyze the data and generate reports. -
It will call the product service, order service, user service and inventory service to get the data and generate reports. -
It will also call the API gateway to get the data and generate reports.
EUREKA-SERVER:
-
This is a service discovery server that allows microservices to register themselves and discover other services. -
It will be used to register the product service, order service, user service and inventory service. -
It will also be used to discover the product service, order service, user service and inventory service.
CONFIGURATION-SERVER:
-
This is a service that provides configuration properties to the microservices. -
It will be used to provide configuration properties to the product service, order service, user service and inventory service. -
It will also be used to provide configuration properties to the API gateway.
Add the @EnableEurekaClient annotation in the main class of the services. Configure the application.yml file with the Eureka server details. Create a new spring boot project for Eureka server and add the dependency (spring-cloud-starter-netflix-eureka-server) in the pom.xml file. Add the @EnableEurekaServer annotation in the main class of the Eureka server. Configure the application.yml file with the Eureka server details. Implement the WebClient in the order service to call the product service.
- create a config class to initialize the WebClient.Builder bean.
- Modify the OrderService class to call ProductService via Eureka Discovery
- Expose an api in order controller to get the product details.
- Run the Eureka server and the services.
TODO: Switch from Spring Data JPA to Spring Data R2DBC by updating your pom.xml org.springframework.boot spring-boot-starter-data-r2dbc io.r2dbc r2dbc-postgresql
- mvn clean install -DskipTests To highlight text in the editor: File → Settings → Editor → TODO→ Add a new pattern and choose a color.
When using WebClient, ensure that you are not creating a new instance of WebClient for each request. Instead, create a single instance of WebClient and reuse it for all requests. This will help in reducing memory leaks and improving performance. Use the WebClient.Builder to create a single instance of WebClient and reuse it for all requests. This will help in reducing memory leaks and improving performance. #What was observed during integration testing: The web application [ROOT] appears to have started a thread named [reactor-http-nio-*] but has failed to stop it. This is very likely to create a memory leak: it is a widespread issue when using Netty-based servers (like Spring Webflux or reactive client like WebClient) in integration tests, especially when tests are run inside Spring boot + Tomcat(Servlet-based) environment. For now, it is being suppressed by setting the following property in the application-test.yml file with: logging: level: org.apache.catalina.loader.WebappClassLoaderBase: ERROR
Use Swagger or Spring REST Docs to document the APIs. Add the dependency in the pom.xml file for Swagger or Spring REST Docs. Configure the application.yml file with the Swagger or Spring REST Docs details. Expose an API to get the API documentation. In this project, we will use springdoc-openapi-ui for API documentation (springdoc.org). Automatically generate doc in JSON/YML and HTML formats APIs. Check more from the link: https://springdoc.org STEPS TO ADD SPRINGDOC-OPENAPI-UI:
- Add the following dependency in the pom.xml file for springdoc-openapi-ui.
- change path in the application.yml: e.g., http://localhost:port/swagger-ui.html
- Add the @OpenAPIDefinition annotation in the main class of the services.
- Configure the application.yml file with the springdoc-openapi-ui details.
- Expose an API to get the API documentation.
- Access the API documentation at http://localhost:port/swagger-ui.html
dependency in the pom.xml file for springdoc-openapi-ui in HTML format. org.springdoc springdoc-openapi-starter-webmvc-ui 2.8.9 dependency in the pom.xml file for springdoc-openapi-ui in JSON/YML format. org.springdoc springdoc-openapi-starter-webmvc-api 2.8.9 command to run docker compose file from the common-utils folder: ****docker compose -f docker-compose.yml up --build command to run docker compose file from the root folder: *****docker compose -f common-utils/docker-compose.yml up --build Check actuator endpoints in the browser: *****http://localhost:port/actuator/health TESTING KAFKA:
PRODUCER:
- docker run -it --rm --network common-utils_microservices-network confluentinc/cp-kafka:latest kafka-console-producer --bootstrap-server kafka:9092 --topic test-topic
After running the above command, you can type messages in the console and press Enter to send them to the Kafka topic. Press Ctrl+C to exit the producer.
CONSUMER:
- docker run --rm --network common-utils_microservices-network confluentinc/cp-kafka:latest kafka-console-consumer --bootstrap-server kafka:9092 --topic ecommerce-topic --from-beginning --max-messages 5
This will show the first 5 messages in test-topic. After running the above command, you will see the messages sent to the Kafka topic in the console. Press Ctrl+C to exit the consumer.
#OR
echo "Hello, Kafka!" | docker run --rm --network common-utils_microservices-network -i confluentinc/cp-kafka:latest kafka-console-producer --bootstrap-server kafka:9092 --topic test-topic Checking specific kafka broker versions:
exec into the kafka container: docker exec -it kafka kafka-broker-api-versions --bootstrap-server localhost:9092 Rebuilding docker image for a specific service with no cache:
docker build --no-cache
docker-compose up -d: To run all images in a docker compose file in detached mode
docker compose up -d --build: To stop all images in a docker compose file
docker compose down: To stop a specific service in a docker compose file
docker compose stop: To start a specific service in a docker compose file
docker compose start: To view logs of a specific service in a docker compose file
docker compose logs -f
docker-compose up -d product-service: To run a specific service in a docker compose file in detached mode
- mvn clean install -DskipTests
- docker build --no-cache *docker compose down -v: to remove all volumes
- docker volume rm common-utils_product-service-data
- docker rm -f
$(docker ps -aq) && docker volume rm $ (docker volume ls -q) && docker rmi -f $(docker images -q)
docker ps -aq → lists all container IDs (running + stopped).
docker rm -f ... → force-removes all those containers.
docker volume rm $(docker volume ls -q)
docker volume ls -q → lists all volume names/IDs.
docker volume rm ... → deletes all volumes.
docker rmi -f $(docker images -q)
docker images -q → lists all image IDs.
docker rmi -f ... → force-removes all images.
&& chaining
Ensures that the next command only runs if the previous one succeeds.
@Container public static final GenericContainer<?> kafka = new GenericContainer<>("confluentinc/cp-kafka:7.8.0") .withExposedPorts(9092, 29092) .withEnv("KAFKA_BROKER_ID", "1") .withEnv("KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR", "1") .withEnv("KAFKA_LISTENERS", "PLAINTEXT://0.0.0.0:9092") .withEnv("KAFKA_ADVERTISED_LISTENERS", "PLAINTEXT://localhost:9092") .waitingFor(Wait.forLogMessage(".started.\n", 1));
Add the dependency in the pom.xml file for FCM. Create a service class to send notifications using FCM. Call the notification service from the order service when an order is placed or updated. More details: https://firebase.google.com/docs/cloud-messaging Example for java: FileInputStream serviceAccount = new FileInputStream("path/to/serviceAccountKey.json"); FirebaseOptions options = new FirebaseOptions.Builder() .setCredentials(GoogleCredentials.fromStream(serviceAccount)) .build(); FirebaseApp.initializeApp(options);
Reasons why .env content gets loaded locally when services run
- When there is a spring boot dev tool dependency in the pom.xml file.
- IDE Automatic .env loading
- Some IDEs (like IntelliJ IDEA) automatically load environment variables from a .env file when running a Spring Boot application.This feature is often enabled by default to facilitate development and testing.
Command to show dependency, for example, com.vaadin.external.google:android-json
- mvn dependency:tree -Dincludes="com.vaadin.external.google:android-json"
How to convert a secret JSON file to a base64 encoded string:
- This is useful when you want to store the secret in an environment variable or a secret manager.
- You can use the following command to convert the JSON file to base64 encoded string:
- Navigate to the directory where the serviceAccountKey.json file is located.
- Run the following command to convert the JSON file to base64 encoded string:
- Linux/Mac: cbase64 firebase-service-account.json > firebase-service-account.json.b64
- Windows (PowerShell): $base64String = [Convert]::ToBase64String([System.IO.File]::ReadAllBytes("firebase-service-account.json")) Set-Content -Path "firebase-service-account.json.b64" -Value $base64String
- Open the firebase-service-account.json.b64 to get the base64 encoded string.
- NOTE: Make sure the JSON file is in the project root folder (if running from there) or provide the correct path to the file.
- Check other details on how to use the secret in GitHub Actions here: ONE NOTE
TODO: For testing there is a new client called RestTestClient, very good for uint, integration and end-end testing Documentation: https://docs.spring.io/spring-framework/reference/7.0-SNAPSHOT/testing/resttestclient.html
use mvn dependency:tree | findstr r2dbc to check if there are any dependencies related to r2dbc in the project.
This command will filter the dependency tree to show only the dependencies that contain "r2dbc" in their name.
If there are any such dependencies, they will be listed in the output. If there are no such dependencies, the output will be empty.
This useful to check if there are any dependencies related to r2dbc in the project, especially after switching from Spring Data JPA to Spring Data R2DBC.
Or if tehre is a conflict between different versions of r2dbc dependencies or if there are any transitive dependencies that are bringing in r2dbc dependencies
into a project that is not using r2dbc.
This command can help identify and resolve such issues by showing the dependencies related to r2dbc in the project.
mvn dependency:tree "-Dincludes=org.springframework.boot:spring-boot-starter-data-r2dbc"
Gives this kind of result:
[INFO] --- dependency:3.8.1:tree (default-cli) @ product-service ---
[INFO] com.tjtechy:product-service:jar:0.0.1
[INFO] \- com.tjtechy:common-utils:jar:0.0.1:compile
[INFO] \- org.springframework.boot:spring-boot-starter-data-r2dbc:jar:3.4.2:compile
product-services uses the JPA(JpaRepository) and the user-service uses R2DBC(R2dbcRepository).
Product-service is beringing the dependency via the common-utils module, so it cannnot start because of the conflict between the JPA and R2DBC dependencies.
- http basics is not used in this project anymore.
- JWT (JSON Web Token) generation and validation using Spring Security OAuth2 Resource Server.
- Role-based access control or authorization.
- Authentication and authorization exception handling.
- Check the flow in the OneNote for more details.
- The oauth2 resource server brings in the:
- oauth2-jose that provides the com.nimbusds.jose library for JWT handling. Check the mvn panel for more details.
- user-service issues the JWTs while the other services validate the JWTs with the JWKS endpoint.
- Use the following command to get the container ID: docker ps
- Use the following command to get a shell inside the container or example postgres: docker exec -it ecommerce-postgres psql -U postgres
- \c "ECommerce-User-Service" to connect to the database inside the container (if not joined by -, no need for quotation if not).
- \d
- \d users: to see the details of the users table.
- q to quit operations inside the container.
- exit to exit the container.
- git rm --cached path/to/file to remove a file from the repository but keep it in the local file system.
- git rm -f --cached path/to/file to force remove a file from the repository but keep it in the local file system.
- mvn clean package -DskipTests
- mvn dependency:tree -Dincludes=commons-io:commons-io
- After upgrading the spring boot version:
- The first approach is to add spring boot properties migrator dependency in the pom.xml (s) file to check for deprecated properties.
- It may be enough to add to the parent pom.xml file, but if there are any issues, add it to the service pom.xml file(s) as well.
- After fixes, the dependency should be removed. It should not be left in production.
- The second approach is to use the IntelliJ plugin to check for deprecated properties.
- In the menu bar, select: code → inspect code → whole project (or select the preferred scope accordingly) → Analyze
- Check the Inspection results for Deprecated properties and warnings.
-Firebase and jwt secret keys are stored in the github secrets and are injected into the services
-via the .env file. The .env file is not committed to the repository for security reasons. During CI
-they are injected into services via the github secrets.
- download the Azure CLI and login to your Azure account.
- create a resource group (use GUI or CLI): -> check under the resource group in the Azure portal for the name of the resource group. Chose your closest location to reduce latency and improve performance.
for example: az group create --name ecommerce-java-resource-group --location swedencentral
resource group is a folder that containes all the resources needed for a project e.g., appplication, keyvaults, Databases, etc.
- create app service plan (use GUI or CLI): -> check under the app service plan in the Azure portal for the name of the app service plan.
for example: az appservice plan create --name ecommerce-java-app-service-plan --resource-group ecommerce-java-resource-group --sku F1 --is-linux
app service plan defines the compute tier for the web apps.
you can specify the sku (pricing tier) for the app service plan. For example, --sku B1 for basic tier.
-create web app (use GUI or CLI): -> check under the web app in the Azure portal for the name of the web app.
for example: az webapp create --name ecommerce-java-web-app --plan ecommerce-java-app-service-plan --resource-group ecommerce-java-resource-group --deployment-container-image-name nginx:latest
web app is the actual application that will be deployed to Azure.
-under App Services in the Azure portal, you can view the newly created web app and its details. You can also view the URL of the web app.
the default domain is : ecommerce-java-web-app.azurewebsites.net
click brouse or copy the URL to view the web app. You will see the default nginx page. Nginx is preloaded on the web page because we used the nginx:latest image for the web app. We will change this to our application image later.
- In Deployment Center,
you can configure the deployment source for the web app.
you can choose to deploy from a local git repository, GitHub, Bitbucket, Azure Repos, etc.
you can also configure the build and deployment settings for the web app.
view logs for the deployment process. You can also view the logs for the web app in the Azure portal.
- You can set a custom domain name under settings for the web app.
- define a deployment workflow in the .github/workflows folder. The workflow will build the application and push the image to the Azure Container Registry (ACR) and then deploy the image to the web app.
deployment to azure is now modified to folow the following steps:
- build the application.
- build the docker image for the application.
- create container app environment in azure.
- deploy
command to check if config properties are being loaded from the config server locally through PowerShell:
- curl.exe http://localhost:8888/api-gateway/default
- In the Run/Debug Configurations dialog, select the service you want to run.
- In the Modify options section, click/enable the Environment variables field.
- Click the folder icon and navigate to the .env file in the project root directory.
- Click OK to save the changes and close the dialog.
- the values in the .env file will be loaded as environment variables when you run the service locally in IntelliJ IDEA.
- az containerapp exec `
--name ecommerce-api-gateway `
--resource-group ecommerce-java-resource-group `
--command "/bin/sh"
- az containerapp delete `
--name ecommerce-api-gateway `
--resource-group ecommerce-java-resource-group `
--yes
currently the browser request via the Azure Portal is reaching the gateway and returning normally without being blocked by CORS
- Create resource
- Choose database → Azure Database for PostgreSQL ...→ Flexible server
- Choose subscription and resource group (e.g., ecommerce-java-resource-group)
- Choose server name (e.g., ecommerce-postgres-db)
- Choose region (e.g., swedencentral)
- Choose version (e.g., 15)
- Choose compute + storage (e.g., Basic, 1 vCore, 5 GB): you need to Configure server here (compute storage)
and choose the pricing tier (e.g., Basic, 1 vCore, 5 GB). You can also choose the backup retention period (e.g., 7 days) and the storage auto-grow option (e.g., Enabled).
- Choose authentication method (e.g., Password)
- Choose username and password (e.g., postgres, yourpassword)
- Choose additional settings (e.g., Enable public access, Allow access to Azure services and prod db using the firewall rules, Enable SSL connection)
- Click Review + create and then Create
- After the deployment is complete, go to the resource and click on Connection strings to get the connection string for your application. Use this connection string in your application.yml file for the database configuration
- open the database resource in the Azure portal
- on the left menu, click connect to see the connection details and connection strings (under connect from your app) for your application in different programming languages (e.g., Java, .NET, Node.js, Python, etc.)
- the jdbc connection string is in the format: jdbc:postgresql://ecommerce-java.postgres.database.azure.com:5432/postgres?user=tjtechy&password={your_password}&sslmode=require is used by the Spring boot application.