通过scala调用python的两种方式

1. 模仿cmd命令方式

使用“Runtime.getRuntime.exec(exeStr)”方法，传入要执行的cmd命令就可以了。

def TestCmdExec(args: Array[String]) {
  var exeStr = "python D:/ckk_software/BDZbiandianzhan_py/scalaExcute.py "
  exeStr += " 李庄10 " + " 0130343451706 "
  var proc = Runtime.getRuntime.exec(exeStr)
  var ir = new InputStreamReader(proc.getInputStream, "GBK")
  var input = new LineNumberReader(ir)
  var result = input.readLine()
  while (result != "match fail" && result != null) {
    println(result)
    result = input.readLine()
  }
  input.close()
  ir.close()
  println(proc.waitFor())
  proc.waitFor()
}

• 优点：操作简单，对于简单的命令什么的可以直接使用
• 缺点：对于批量调用python程序比较复杂，也比较愚蠢，关于路径什么的都需要修改，不建议使用。

2. 编写restful接口，在scala内部发送http请求来调用

restful服务：URL定位资源，用HTTP动词（GET,POST,DELETE,DETC）描述操作。

1. 编写接口

因为要调用的程序是python程序，所以要使用python的flask来编写接口。

from function import *
import pandas as pd
from flask import Flask,url_for,request
import json

data = pd.ExcelFile("cidain2.xlsx")
data = data.parse("Sheet1")
cidian = data['PMS_LINE_NAME'].values.tolist()
org_no = data['PI_ORG_NO'].values.tolist()
id = data['PMS_LINE_ID'].values.tolist()
# go方法为需要scala端调用的程序
def go(text,org_id,cidian,org_no,id):
    '''
    :param text: "高庄10"
    :param org: 130343451706
    :return:
    '''
    org_type = "04"
    if org_type == 3:
        org = str(org_id)[:-2]
    else:
        org = str(org_id)
    label, cidian, id = fun(org, text, cidian, org_no, id)
    label, id = filt(label, cidian, id)
    if len(label) > 0 :
        for i in range(0,len(label)):    
            print(label[i]+"\t"+id[i])
    else :
        print("match fail")
    return "\t".join(label)+"\t"+"\t".join (id)
# 注册flaskapp,接收get请求进行处理
app = Flask(__name__)
# get请求参数在url中，访问“http：//127.0.0.1：5000/reason/orgId”
@app.route("/<reason>/<orgId>")
def exeFlask(reason,orgId): 
    preResult = go(reason,orgId)
    return str(preResult)
# 接收post请求，请求参数封装在客户端的request中请求中
# 访问“http：//127.0.0.1：5000/”
@app.route('/',methods=['POST'])
def postRestful():
    data = json.loads(str(request.data,'utf-8'))
    reason = data['reason']
    orgId =data['orgId'] 
    print("reason: " + reason + "\torgId:\t" + orgId)
    preResult = go(reason,orgId,cidian,org_no,id)
    return str(preResult)
# 启动接口，run方法内可以不设参数，默认本地的5000端口
# 如果要允许同内网访问的话要把host设置为“0.0.0.0”
app.run(host="0.0.0.0",port="5000")

2. scala端发送请求（httpclient4.5.5和httpclient3.0.1两种版本）

• 前言：httpclient4和httpclient3的差别还是蛮大的，所以使用需要注意版本的问题，而且就网上查阅的资料来看httpclient和httpcore的版本冲突还是蛮多的，还有和spark的版本兼容性问题也是。

1. httpclient4.5.5

• httpclient4.5.5是用在spark2.3上面的，maven添加依赖方法略

  get请求示例

def TestHttpGet(url: String): String = {
   import org.apache.http.client.methods.{ HttpGet }
   import org.apache.http.impl.client.HttpClients
   import org.apache.http.util.EntityUtils

   var host = "http://10.138.72.199:5000/"
   var orgId = "/0130343451006"
   var rawText = "_霍邱县开发区杭庙村石船 
   var url = host + rawText + orgId
   var HttpClient = HttpClients.createDefault()
   val get = new HttpGet(url)
   var response = HttpClient.execute(get)
   EntityUtils.toString(response.getEntity)
 }

post请求示例

def restful(){
    import org.apache.http.{ HttpEntity, HttpResponse }
    import org.apache.http.client.methods.HttpPost
    import org.apache.http.entity.StringEntity
    import org.apache.http.impl.client.DefaultHttpClient
    import org.apache.http.protocol.HTTP
    import org.apache.http.util.EntityUtils
    val httpclient = new DefaultHttpClient
    try {
      val url = "http://10.138.72.199:5000"
      val reason = "_池河大道交警队处_交警队外台区新架设低压电缆释放"
      val orgId = "0130343451304" //经度
      val dtype = "json" //返回数据的格式,xml或json，默认json
      val jsonParams = "{\"reason\":\"" + reason + "\",\"orgId\": \"" +orgId + "\",\"dtype\":\"" + dtype + "\"}"
     
      //创建
      val httpost = new HttpPost(url)
      println("参数：" + jsonParams)
      //json字符串,准备发送

      try {
        val stringEntity = new StringEntity(jsonParams, HTTP.UTF_8)
        stringEntity.setContentType("application/json;charset=utf-8")
        httpost.setEntity(stringEntity)
        var response: HttpResponse = null
        response = httpclient.execute(httpost)
        val entity: HttpEntity = response.getEntity
        try {
          //必需触发
          println(EntityUtils.toString(entity))
        } catch {
          case appleException: Exception => println(appleException.toString)
        }
      } catch {
        case e: Exception => {
          println("Exception is " + e.toString)
        }
      }
    } catch {
      case sendE: Exception => { throw sendE }
    } finally {
      //关闭
      httpclient.getConnectionManager.shutdown
    }
  }

2.httpclient3.0.1

• httpclient3主要是支持spark1.6使用的，但是在spark2.3上面也同样兼容，但是httpclient4不支持spark1.6，因为手上的项目是spark1.6的，所以需要使用httpclient3。而且httpclient3的版本也比较古老，所以网上很少可以用的参考资料。这里是参考java版本写的scala版本的调用接口。

  依赖

<dependency>
		<groupId>commons-httpclient</groupId>
		<artifactId>commons-httpclient</artifactId>
		<version>3.0.1</version>
	</dependency>

post客户请求方法（这里因为只需要用到post方法就只写了这个版本的）

def TestHttpClient3(){
      import org.apache.commons.httpclient._
      import org.apache.commons.httpclient.methods._
   
      val client = new HttpClient
//      val method = new PostMethod("http://10.138.72.199:5000")
      val method = new PostMethod("http://127.0.0.1:5000")
      val reason = "_徽州区富溪乡_田里3817台区改造：横担金具安装、拉线制作，低压导线展放等。"
      val orgId = "01303434512" //orgId
      val dtype = "json" //返回数据的格式,xml或json，默认json
      val jsonParams = "{\"reason\":\"" + reason + "\",\"orgId\": \"" + orgId + "\",\"dtype\":\"" + dtype + "\"}"
      //设置传输头文件和json参数
      println(jsonParams)
      method.setRequestHeader("Content-Type","application/json")
      method.setParameter("requestdata", jsonParams)
      val se = new StringRequestEntity(jsonParams,"application/json","UTF-8")
      method.setRequestEntity(se)
      //执行请求
      val statusCode = client.executeMethod(method)
      print(statusCode)
      //获取返回的结果
      val responseBody = method.getResponseBodyAsStream()
      val resp = IOUtils.toString(responseBody,"utf8")
      println(resp)
      //关闭连接
      method.releaseConnection()
    }

python服务器端

//创建连接
#app = Flask(__name__)
#@app.route('/',methods=['POST'])
#def postRestful():
#   。。。。。。
#  此处为要执行的python程序体
#
//开启服务，0.0.0.0表示允许局域网访问，5000是端口
#app.run(host="0.0.0.0",port="5000")