Note: this is not about how to build a hadoop development environment in Windows and how to ensure that it runs . I hope you can see the problem clearly first, don"t spray me, thank you
< hr > < H2 > problem description < / H2 >recently set up a hadoop cluster in the virtual machine, and configured the hadoop-eclipse-plugin plug-in to connect to hadoop, on the Windows through eclipse for MapReduce-related programming development. However, one problem is that the MapReduce program running on Win is obviously much faster than in the virtual machine . I don"t know why?
secondly, as shown in the figure. I typed the program into a jar package before compilation, and the program also specified the use of the jar package, and the program ran normally. However, I have a doubt that I did not manually specify where the jar package was uploaded to the virtual machine , nor did I set up shared files or anything else . Why does hadoop still find and call the jar package and execute the program ? And the jar package was not found in the virtual machine [I don"t know and didn"t upload it, or I don"t know where it was uploaded]
- my initial guess is that developers under windows will download a separate set of jar packages that hadoop depends on. Is it possible that the simulation is just running under Windows and running in the hadoop cluster environment, while the local host is actually running
- but I checked JobHistory, through web and found that had a record of datanode node jobs and ruled out the above idea
- can confirm that the program must be running in a virtual machine, so you must have used the jar package mentioned above [but I don"t know how the jar package was uploaded? Also can"t find]
- guess if the plug-in hadoop-eclipse-plugin helped me complete some submission process? Or something?
there are only two questions above, why is it faster? How can I find the jar package? Have you uploaded it?
I hope there are bosses who know one or two and can provide help and help me solve my doubts. Thank you!