问题描述
各位好，初学Spark，想让Spark从S3上的数据Spark搭建在EC2上我在spark-shell下执行：sc.hadoopConfiguration.set("fs.s3a.access.key","xxxx")sc.hadoopConfiguration.set("fs.s3a.secret.key","yyyy")valtextFile=sc.textFile("s3a://...")textFile.count()异常：15/12/2113:29:58INFOS3AFileSystem:CaughtanAmazonServiceException,whichmeansyourrequestmadeittoAmazonS3,butwasrejectedwithanerrorresponseforsomereason.15/12/2113:29:58INFOS3AFileSystem:ErrorMessage:StatusCode:403,AWSService:AmazonS3,AWSRequestID:A5C5253A63B271A6,AWSErrorCode:null,AWSErrorMessage:Forbidden15/12/2113:29:58INFOS3AFileSystem:HTTPStatusCode:40315/12/2113:29:58INFOS3AFileSystem:AWSErrorCode:null15/12/2113:29:58INFOS3AFileSystem:ErrorType:Client15/12/2113:29:58INFOS3AFileSystem:RequestID:A5C5253A63B271A615/12/2113:29:58INFOS3AFileSystem:ClassName:com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exceptioncom.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception:StatusCode:403,AWSService:AmazonS3,AWSRequestID:A5C5253A63B271A6,AWSErrorCode:null,AWSErrorMessage:Forbidden,S3ExtendedRequestID:7zaafP60a1U8pt02jayWuWtf1ONVd1g5n6/xAbutkxtU/M1ZvtSV0pjVzPN2aFOpANTj5R7Cikg=atcom.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:798)atcom.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:421)atcom.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)atcom.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)看起来是被服务器拒绝（403）了请问是不是我操作哪里错了？是否跟北京区S3有关，默认访问了global的？Endpoint／Region？另外有链接给我学习一下s3://、s3n://、s3a://的区别吗？我该用哪个？谢谢！