2424人阅读
& & 最近在使用Flume收集日志，发现每隔几个小时就flume进程就会挂掉，但是日志中又没有Error 和Exception，进程结束前的一些日志如下：
09 Apr :21,160 INFO
[agent-shutdown-hook] (org.apache.flume.lifecycle.LifecycleSupervisor.stop:79)
- Stopping lifecycle supervisor 11
09 Apr :21,162 INFO
[agent-shutdown-hook] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider.stop:83)
- Configuration provider stopping
09 Apr :21,177 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.Log.roll:932)
- Roll start /maichuang/flume/channel/data
09 Apr :21,177 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.LogFile$Writer.&init&:214)
- Opened /maichuang/flume/channel/data/log-15
09 Apr :21,178 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.Log.roll:948)
- Roll end
09 Apr :21,210 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.Log.roll:932)
- Roll start /maichuang/flume/channel/data
09 Apr :21,211 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.LogFile$Writer.&init&:214)
- Opened /maichuang/flume/channel/data/log-16
09 Apr :21,211 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.channel.file.Log.roll:948)
- Roll end
09 Apr :21,211 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)
- Unable to deliver event. Exception follows.
org.apache.flume.ChannelException: Commit failed due to IO error [channel=channel1]
at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:605)
at org.apache.flume.channel.BasicTransactionSemantics.rollback(BasicTransactionSemantics.java:168)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:466)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:380)
at org.apache.flume.channel.file.LogFileV3.writeDelimitedTo(LogFileV3.java:148)
at org.apache.flume.channel.file.LogFileV3$Writer.&init&(LogFileV3.java:209)
at org.apache.flume.channel.file.LogFileFactory.getWriter(LogFileFactory.java:77)
at org.apache.flume.channel.file.Log.roll(Log.java:935)
at org.apache.flume.channel.file.Log.roll(Log.java:904)
at org.apache.flume.channel.file.Log.rollback(Log.java:719)
at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:603)
... 5 more
09 Apr :21,212 INFO
[agent-shutdown-hook] (org.apache.flume.sink.hdfs.HDFSEventSink.stop:499)
- Closing hdfs://master:9000/flume/-1446.log
09 Apr :21,213 INFO
[agent-shutdown-hook] (org.apache.flume.sink.hdfs.BucketWriter.close:409)
- Closing hdfs://master:9000/flume/_-1446.log.6.tmp
09 Apr :21,214 INFO
[hdfs-master-call-runner-9] (org.apache.flume.sink.hdfs.BucketWriter$3.call:339)
- Close tries incremented& & 虽然我不太懂java，但是我还是硬着头皮去源码里面查看&pre name=&code& class=&java&&org.apache.flume.lifecycle.LifecycleSupervisor
对应的源码，感觉应该不是这里的问题。后来，感觉应该找 &agent-shutdown-hook 相关的信息，果然找到了：
//apache/flume/HEAD/flume-ng-node/src/main/java/org/apache/flume/node/Application.java?line=345
public static void main(String[] args) {
boolean isZkConfigured =
Options options = new Options();
Option option = new Option(&n&, &name&, true, &the name of this agent&);
option.setRequired(true);
options.addOption(option);
option = new Option(&f&, &conf-file&, true,
&specify a config file (required if -z missing)&);
option.setRequired(false);
options.addOption(option);
option = new Option(null, &no-reload-conf&, false,
&do not reload config file if changed&);
options.addOption(option);
// Options for Zookeeper
option = new Option(&z&, &zkConnString&, true,
&specify the ZooKeeper connection to use (required if -f missing)&);
option.setRequired(false);
options.addOption(option);
option = new Option(&p&, &zkBasePath&, true,
&specify the base path in ZooKeeper for agent configs&);
option.setRequired(false);
options.addOption(option);
option = new Option(&h&, &help&, false, &display help text&);
options.addOption(option);
CommandLineParser parser = new GnuParser();
CommandLine commandLine = parser.parse(options, args);
if (commandLine.hasOption('h')) {
new HelpFormatter().printHelp(&flume-ng agent&, options, true);
String agentName = commandLine.getOptionValue('n');
boolean reload = !commandLine.hasOption(&no-reload-conf&);
if (commandLine.hasOption('z') || commandLine.hasOption(&zkConnString&)) {
isZkConfigured =
Application application =
if (isZkConfigured) {
// get options
String zkConnectionStr = commandLine.getOptionValue('z');
String baseZkPath = commandLine.getOptionValue('p');
if (reload) {
EventBus eventBus = new EventBus(agentName + &-event-bus&);
List&LifecycleAware& components = Lists.newArrayList();
PollingZooKeeperConfigurationProvider zookeeperConfigurationProvider =
new PollingZooKeeperConfigurationProvider(
agentName, zkConnectionStr, baseZkPath, eventBus);
components.add(zookeeperConfigurationProvider);
application = new Application(components);
eventBus.register(application);
StaticZooKeeperConfigurationProvider zookeeperConfigurationProvider =
new StaticZooKeeperConfigurationProvider(
agentName, zkConnectionStr, baseZkPath);
application = new Application();
application.handleConfigurationEvent(zookeeperConfigurationProvider
.getConfiguration());
File configurationFile = new File(commandLine.getOptionValue('f'));
* The following is to ensure that by default the agent will fail on
* startup if the file does not exist.
if (!configurationFile.exists()) {
// If command line invocation, then need to fail fast
if (System.getProperty(Constants.SYSPROP_CALLED_FROM_SERVICE) ==
String path = configurationFile.getPath();
path = configurationFile.getCanonicalPath();
} catch (IOException ex) {
logger.error(&Failed to read canonical path for file: & + path,
throw new ParseException(
&The specified configuration file does not exist: & + path);
List&LifecycleAware& components = Lists.newArrayList();
if (reload) {
EventBus eventBus = new EventBus(agentName + &-event-bus&);
PollingPropertiesFileConfigurationProvider configurationProvider =
new PollingPropertiesFileConfigurationProvider(
agentName, configurationFile, eventBus, 30);
components.add(configurationProvider);
application = new Application(components);
eventBus.register(application);
PropertiesFileConfigurationProvider configurationProvider =
new PropertiesFileConfigurationProvider(
agentName, configurationFile);
application = new Application();
application.handleConfigurationEvent(configurationProvider
.getConfiguration());
application.start();
final Application appReference =
&strong&&span style=&color:#ff0000;&&Runtime.getRuntime().addShutdownHook(new Thread(&agent-shutdown-hook&) {
public void run() {
appReference.stop();
});&/span&&/strong&
} catch (Exception e) {
logger.error(&A fatal error occurred while running. Exception follows.&,
& & 上面红色代码部分应该就是结束进程的代码所在。
& & 于是，我又查找&Runtime.getRuntime().addShutdownHook，在
//openjdk-mirror/jdk7u-jdk/master/src/share/classes/java/lang/Runtime.java?keyword=getRuntime&line=209
中找到了：
public void addShutdownHook(Thread hook) {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
sm.checkPermission(new RuntimePermission(&shutdownHooks&));
ApplicationShutdownHooks.add(hook);
}//openjdk-mirror/jdk7u-jdk/HEAD/src/share/classes/java/lang/ApplicationShutdownHooks.java?line=83中也有相关的代码：
static synchronized void add(Thread hook) {
if(hooks == null)
throw new IllegalStateException(&Shutdown in progress&);
if (hook.isAlive())
throw new IllegalArgumentException(&Hook already running&);
if (hooks.containsKey(hook))
throw new IllegalArgumentException(&Hook previously registered&);
hooks.put(hook, hook);
}另外在Application.java源码中，有这么一段：
public synchronized void stop() {
supervisor.stop();
if(monitorServer != null) {
monitorServer.stop();
然后我又去看&supervisor.stop()对应的源码，可从Application.java 中
public Application(List&LifecycleAware& components) {
supervisor = new LifecycleSupervisor();
可知：supervisor = new LifecycleSupervisor()
supervisor 是 &LifecycleSupervisor的一个实例，故又转而看LifecycleSupervisor所在的源码：
//apache/flume/HEAD/flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java?line=51
Application.java代码中的
application.start();
&span style=&color:#ff0000;&&//启动application之后再执行如下代码？？这样不就是一直会执行stop()了么？这中间的逻辑我还没搞明白，期待高手帮忙解答。&/span&
final Application appReference =
Runtime.getRuntime().addShutdownHook(new Thread(&agent-shutdown-hook&) {
public void run() {
appReference.stop();
});如上面的红色字体，目前还没搞懂，望高手留言指点迷津，或发我邮箱 & ,谢谢！
&&相关文章推荐
* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
访问：336799次
积分：4141
积分：4141
排名：第7827名
原创：71篇
转载：113篇
评论：24条
(3)(1)(1)(1)(4)(9)(1)(3)(2)(4)(3)(2)(2)(1)(1)(1)(3)(1)(1)(3)(8)(8)(4)(11)(1)(4)(11)(20)(35)(11)(1)(13)(10)
(window.slotbydup = window.slotbydup || []).push({
id: '4740881',
container: s,
size: '200,200',
display: 'inlay-fix'XMPP本地Mysql无法启动
最近在本地运行xmpp的时候，偶尔遇到了mysql罢工，具体原因不明。但是在网上找到了解决的方法
18:08:34 &[mysql] Error: MySQL shutdown
unexpectedly.
18:08:34 &[mysql] This may be due to a
blocked port, missing dependencies,&
18:08:34 &[mysql] improper privileges, a
crash, or a shutdown by another method.
18:08:34 &[mysql] Press the Logs button to
view error logs and check
18:08:34 &[mysql] the Windows Event Viewer
for more clues
18:08:34 &[mysql] If you need more help, copy
and post this
18:08:34 &[mysql] entire log window on the
其中有位网友提到他的解决方法：
exit&Xampp server
go to your&C:\xampp\mysql\data&directory
delete&the&ibdata1&file
restart&xampp server
已投稿到：
以上网友发言只代表其个人观点，不代表新浪网的观点或立场。