1. There is no problem with the local test, and there is a problem that you cannot connect to the formal environment when you go online. 2. Now install all the hadoop clusters on one machine and rule out the network reasons. ...
1. In the hadoop environment built, datanode uses private network ip (172.16.1.142172.16.1.148), and only 142nodes activate external network ip . 2. The data on the HDFS is obtained remotely through Java, and the datanode address returned to the Java p...
first of all, to introduce my situation, I have a physical machine as a master node, which is referred to as master later. There are also two additional servers, later known as node1 and node2. Where the docker node slave1-slave10, is configured on node...
I have a batch of data (10 billion) as follows, ID FROM TO 1 A B 2 A C 3 B A 4 C A Delete duplicate two-way relational data as follows ID FROM TO 1 A B 2 A C 1. Because the amount of data is too large, bloomfilter is no...
the write orc file code is: public static <T> void write(Class cls, List<T> datas,Properties props) throws Exception{ String path = props.getProperty("localTempFile"); JobConf conf = new JobConf(); FileSystem fs = F...
problem description hive is stuck or has no speed while executing mapreduce read the log hadoop resourcemanager reported an error 2018-07-17 19br 17br the environmental background of the problems and what methods you have tried set up a hadoop c...
< H1 > hadoop startup error < H1 > configuration files are all written and then started [root@master logs]-sharp start-dfs.sh Starting namenodes on [master] Last login: Wed Jul 4 16:30:19 CST 2018 on pts 0 bash v3.2+ is required. Sorry. Starting da...
< H2 > Business scenario < H2 > A large number of json files need to be read and re-parsed and imported into elasticsearch . Json files are saved in different date folders. The size of a single folder is about 80g. The number of json files under the ...
this is the article I refer to. The program that runs according to this article https: blog.csdn.net wangato., as shown in the picture, always reports the error in the first sentence. I Baidu has the reason, but has not been able to solve it. 1. Py...
introduce the virtual machine environment, which is in the spark cluster. The virtual machine is the master node. After the virtual machine is restarted abnormally, the vnc of other nodes can log in normally, which should have nothing to do with the spa...
main server jps shows DataNodeDataNode jps: dead nodes is displayed as 0 ...
in the hadoop environment built by the company, datanode uses the private network ip,. Only the master node has enabled the external network ip, now the data on the HDFS is remotely obtained through Java, and the datanode address returned by master to t...
use a simple java client to send a message after simulating the configuration of a kafka server, but without message storage, you can only see the error log all the time (the message is really not stored) kafka.common.KafkaException: Wrong request type...
when running hadoop streaming, an error is reported. The exception content is as follows: Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 126 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads...
1. First of all, the zeppelin has been started successfully on the terminal 2. Secondly, the port number of the configuration information is not occupied is 8080 3, but the browser enters http: localhost:8080 , but it cannot be accessed. The error h...
uses the scrapy.Request method to collect pages, but nothing is done. import scrapy def ret(response): print( start print ) print(response.body) url = https: doc.scrapy.org en latest intro tutorial.html v = scrapy.http.Request(url=url,...
WeChat Mini Programs swiper slider view container, : I wonder if they use swiper implementations ...
sign the electronic signature with the three parties, and we provide the template. After the three parties download it, sign the electronic signature first, and then send it back to us, because the electronic signature is automatic and will not check the...
regular matching string: "Registrant Name: huangruihong Registrant Organization: huangruihong Registrant Street: xi cheng qu Registrant City: BeiJing Registrant State Province: BJ Registrant Postal Code: 100089 Registrant Country: CN Registrant Phone: +...
SELECT ss.quan FROM `stock` as ss JOIN `order_record` as r ON r.order_id = ss.order_id WHERE ss.from = c AND r.from = c AND r.status = ing AND ss.prod_id = 120 GROUP BY ss.order_id ...