Spring Cloud Dataflow入门

1、下载下面的文件

#Kafka 0.10.1.1
http://kafka.apache.org/downloads
#服务端
http://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-server-local/1.1.3.RELEASE/spring-cloud-dataflow-server-local-1.1.3.RELEASE.jar
#客户端
http://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-shell/1.1.3.RELEASE/spring-cloud-dataflow-shell-1.1.3.RELEASE.jar
#Avogadro.properties
http://bit.ly/Avogadro-SR1-stream-applications-kafka-10-maven

2、启动服务

#启动ZK
bin/zookeeper-server-start.sh config/zookeeper.properties  &
#启动kafka
bin/kafka-server-start.sh config/server.properties &
#启动服务端
java -jar spring-cloud-dataflow-server-local-1.1.3.RELEASE.jar &
#启动客户端
java -jar spring-cloud-dataflow-shell-1.1.3.RELEASE.jar
#导入Avogadro.properties
app import --uri file://PATH_TO_FILE/Avogadro.properties

3、打开界面
http://IP:9393/dashboard

4、新建一个简单的流,每15秒,写入一行数据
就不截图了,直接粘进去就好了

per_15s_flow01=per_15s: time --cron="0/15 * * * * ?" --date-format="yyyy-MM-dd HH:mm:ss" | file01: file --directory=PATH_TO_FILE --name=file01.txt

创建,并开启工作流。
查看file01.txt
销毁工作流。

5、新建一个较为复杂的流

per_15s_flow01=per_15s: time --cron="0/15 * * * * ?" --date-format="yyyy-MM-dd HH:mm:ss" | file01: file --directory=PATH_TO_FILE --name=file01.txt
per_15s_subflow01=:per_15s_flow01.per_15s > groovy-transform --script="nscript.groovy" --variables=msg='Time is ' | file02: file --directory=PATH_TO_FILE/ --name=file02.txt

用到了一个脚本nscript.groovy

msg+payload

创建,并开启工作流。
查看file01.txt及file02.txt
销毁工作流。

如果报错找不到文件nscript.groovy的话,可以考虑把nscript.groovy文件加到groovy-transform-processor-kafka-10-1.1.1.RELEASE.jar中。

Spring Cloud Config入门

1、从github上下载最新版本
spring-cloud-config

2、编译并运行spring-cloud-config-server

cd spring-cloud-config-server
../mvnw spring-boot:run

如果运行mvnw遇到下面的错误,用vi处理一下就好了

/bin/sh^M: bad interpreter: No such file or directory

用vi处理一下

:set fileformat=unix

3、默认端口为8888,默认配置仓库为github
config-repo

支持以下访问语法:

#获取配置
/{application}/{profile}[/{label}]
/{application}-{profile}.properties
/{label}/{application}-{profile}.properties
/{application}-{profile}.yml
/{label}/{application}-{profile}.yml

#更新Environment、重新绑定@ConfigurationProperties、重置日志级别
/env
#刷新@RefreshScope
/refresh
#重启Spring上下文,默认是禁用的
/restart` for restarting the Spring context (disabled by default)
#调用ApplicationContext的stop及start等生命周期相关函数
/pause
/resume

我们来测试一下服务端运行情况:(服务端要可以连接到github上哦)

#foo.properties
curl localhost:8888/foo-master.properties
#foo.properties tag=v1.0.0.RC2
curl localhost:8888/v1.0.0.RC2/foo-master.properties
#foo-development.properties
curl localhost:8888/foo-development.properties
#foo-db.properties
curl localhost:8888/foo-db.properties
#foo-development-db.properties
curl localhost:8888/foo-development-db.properties

4、看一下自带的例子
spring-cloud-config-sample/src/main/java/sample/Application.java


package sample;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@SpringBootApplication
public class Application {

	@Autowired
	private Environment environment;

	@Value("${spring.application.name}")
	private String appName;
	
	@RequestMapping("/appName")
	public String appName() {
		return appName;
	}

	@RequestMapping("/")
	public String query(@RequestParam("q") String q) {
		return environment.getProperty(q);
	}

	public static void main(String[] args) {
		SpringApplication.run(Application.class, args);
	}

}

5、测试一下例子

cd spring-cloud-config-sample
mvn spring-boot:run

6、看一下输出信息

Mapping servlet: 'dispatcherServlet' to [/]
Mapping filter: 'metricsFilter' to: [/*]
Mapping filter: 'characterEncodingFilter' to: [/*]
Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
Mapping filter: 'httpPutFormContentFilter' to: [/*]
Mapping filter: 'requestContextFilter' to: [/*]
Mapping filter: 'webRequestLoggingFilter' to: [/*]
Mapping filter: 'applicationContextIdFilter' to: [/*]
Mapping : Mapped "{[/]}" onto public java.lang.String sample.Application.query(java.lang.String)
Mapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
Mapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
Mapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
Mapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
Mapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
Mapping     : Mapped "{[/dump || /dump.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/auditevents || /auditevents.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public org.springframework.http.ResponseEntity<?> org.springframework.boot.actuate.endpoint.mvc.AuditEventsMvcEndpoint.findByPrincipalAndAfterAndType(java.lang.String,java.util.Date,java.lang.String)
Mapping     : Mapped "{[/loggers/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.LoggersMvcEndpoint.get(java.lang.String)
Mapping     : Mapped "{[/loggers/{name:.*}],methods=[POST],consumes=[application/vnd.spring-boot.actuator.v1+json || application/json],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.LoggersMvcEndpoint.set(java.lang.String,java.util.Map<java.lang.String, java.lang.String>)
Mapping     : Mapped "{[/loggers || /loggers.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/env],methods=[POST]}" onto public java.lang.Object org.springframework.cloud.context.environment.EnvironmentManagerMvcEndpoint.value(java.util.Map<java.lang.String, java.lang.String>)
Mapping     : Mapped "{[/env/reset],methods=[POST]}" onto public java.util.Map<java.lang.String, java.lang.Object> org.springframework.cloud.context.environment.EnvironmentManagerMvcEndpoint.reset()
Mapping     : Mapped "{[/pause || /pause.json],methods=[POST]}" onto public java.lang.Object org.springframework.cloud.endpoint.GenericPostableMvcEndpoint.invoke()
Mapping     : Mapped "{[/resume || /resume.json],methods=[POST]}" onto public java.lang.Object org.springframework.cloud.endpoint.GenericPostableMvcEndpoint.invoke()
Mapping     : Mapped "{[/beans || /beans.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/configprops || /configprops.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/features || /features.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/env/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EnvironmentMvcEndpoint.value(java.lang.String)
Mapping     : Mapped "{[/env || /env.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/health || /health.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(javax.servlet.http.HttpServletRequest)
Mapping     : Mapped "{[/restart || /restart.json],methods=[POST]}" onto public java.lang.Object org.springframework.cloud.context.restart.RestartMvcEndpoint.invoke()
Mapping     : Mapped "{[/autoconfig || /autoconfig.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/heapdump || /heapdump.json],methods=[GET],produces=[application/octet-stream]}" onto public void org.springframework.boot.actuate.endpoint.mvc.HeapdumpMvcEndpoint.invoke(boolean,javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException,javax.servlet.ServletException
Mapping     : Mapped "{[/metrics/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.MetricsMvcEndpoint.value(java.lang.String)
Mapping     : Mapped "{[/metrics || /metrics.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/mappings || /mappings.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/refresh || /refresh.json],methods=[POST]}" onto public java.lang.Object org.springframework.cloud.endpoint.GenericPostableMvcEndpoint.invoke()
Mapping     : Mapped "{[/info || /info.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
Mapping     : Mapped "{[/trace || /trace.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()

7、测试一下

curl localhost:8080/?q=foo

8、是不是很简单,到自己的项目上试试吧

Spring Cloud各模块作用

首先说明一下,由于工作实在太忙了,这个系列的后面几篇,是后面补的,并没能及时进行总结和整理。
有一些细节的地方,可能并不准确(调整过部分代码和配置文件什么的),遇到问题麻烦调整一下,有问题也可以留言。

1、配置中心(基于git、svn、ftp等方式)
Spring Cloud Config

2、负载均衡
Spring Cloud Cluster

3、消息总线
Spring Cloud Bus

4、服务注册及发现
Spring Cloud Zookeeper
Spring Cloud Consul

5、任务调度
Spring Cloud Data Flow 任务调度
Spring Cloud Stream 工作流
Spring Cloud Stream App Starters 工作流实现
Spring Cloud Task 批处理
Spring Cloud Task App Starters 批处理实现

6、服务监控
Spring Cloud Sleuth

7、安全(OAuth2)
Spring Cloud Security

8、数据库、消息队列等
Spring Cloud Connectors

9、快速开发groovy
Spring Cloud CLI

10、Moke测试
spring-cloud-contract

11、部署(现在有mvn和docker两种方式)
spring-cloud-deployer

12、第三方云厂商集成
Spring Cloud for Amazon Web Services
Spring Cloud for Cloud Foundry
Spring Cloud Cloud Foundry Service Broker

13、Spring Cloud Netflix
archaius 配置管理
ribbon 客户端负载均衡
feign 基于注解的JSON库,自带ribbon功能
eureka 服务注册、LB、故障转移
hystrix 断路器/日志采集
turbine 性能日志聚合
Zuul API网关,数据路由+负载均衡
spectator/servo 客户端性能指标采集
atlas 时间序列数据库

14、其他
sidecar 将非JVM服务,封装为Netflix服务,并提供代理,让非JVM服务调用

15、移动端支持
RxJava 响应式编程支持(observable, operator和susbscriber)