Skip to content

网络(Network)

通信机制

  • RPC(Remote Procedure Call):Protocol Buffer、Thrift、Avro(IDL、Serialization)
  • Message:Queue、Pub-Sub(Pull、Push)
  • Multi Broadcast:Gossip(Best Effort、Anti-Entropy、Rumor Mongering)

角色(Role)

Master, Worker, Client, Driver, Executor

RPC

基于Netty

  • Context(上下文:Local、Remote)
  • Env(运行环境)
  • Endpoint(终端)

主要作用

  • 消息互通:Event、Status
  • 文件传输:Fetch、Upload
  • Block:Store、Replication
  • Shuffle:Writer、Reader

Reference