Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging staging to master (Release 2.0) #41

Merged
merged 50 commits into from
Aug 6, 2015
Merged

Merging staging to master (Release 2.0) #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
5df9df6
Fixed bug in the story #89696224 for downloading human tagged tweets
kushalkantgoyal Jul 21, 2015
21ba254
Added a script for running the testers.
kushalkantgoyal Jul 24, 2015
88ac8ee
Finished #96994368 Check directories when reading the configuration file
kushalkantgoyal Jul 24, 2015
ae6e256
aidr-analytics debugging: WIP
koushiksinha Jul 27, 2015
b867f7f
aidr-analytics bug fix - needs testing; refactored to removed aidr-ou…
koushiksinha Jul 27, 2015
42b3367
reduced debug msgs
koushiksinha Jul 27, 2015
22faeb5
Minor changes
kushalkantgoyal Jul 28, 2015
b806140
Merge branch 'dev' of https://github.com/qcri-social/AIDR into dev
kushalkantgoyal Jul 28, 2015
f609a99
Commit for #98777058
Jul 28, 2015
3515fbd
Commit for #98777744
Jul 28, 2015
02b4917
updated code to address Latika's issue - needs testing by Latika; 2 n…
koushiksinha Jul 28, 2015
396662e
updated documentation of 2 new REST APIs;reduced debug msgs
koushiksinha Jul 28, 2015
da277bf
Merge branch 'dev' of https://github.com/qcri-social/AIDR into dev
koushiksinha Jul 28, 2015
a5bfee7
aidr-analytics config data cleanup; modified single tweet injection i…
koushiksinha Jul 29, 2015
fce88bb
changed json to return unix time for startTime & endTime
koushiksinha Jul 29, 2015
8216b33
Sending mails on errors in ajax request done.
kushalkantgoyal Jul 29, 2015
55e1e01
Merge branch 'dev' of https://github.com/qcri-social/AIDR into dev
kushalkantgoyal Jul 29, 2015
5aaf70c
Add a new api to send email
kushalkantgoyal Jul 29, 2015
8d73fe7
Commit for #98777744
Jul 30, 2015
a7df701
Commit for #83021780 : Remove ErrorLog from aidr-tagger-api, aidr-tag…
Jul 30, 2015
5d00fce
Minor changes
kushalkantgoyal Jul 30, 2015
c82ad49
Merge branch 'dev' of https://github.com/qcri-social/AIDR into dev
kushalkantgoyal Jul 30, 2015
9336bf9
Commit for #83021780 : Remove ErrorLog from aidr-trainer-api
Jul 30, 2015
78d4164
Sending mail if schedule task failed in Manager
kushalkantgoyal Jul 30, 2015
465e292
Commit for #83021780 : Remover ErrorLog usage from aidr-common, aidr-…
Jul 30, 2015
f193f66
Reverting changes for deploy.sh and config.properties
Jul 30, 2015
1045070
Logging in AIDR-Output
kushalkantgoyal Jul 31, 2015
09824be
Commit for #98777594 : aidr-collector send email alert if not functio…
Jul 31, 2015
c8e52a9
Added logs in AIDR-Persister.
kushalkantgoyal Jul 31, 2015
25dcdaa
Commit for #97809356 : logging for aidr-analytics module
Jul 31, 2015
0c93814
Added logs in AIDR-Collector.
kushalkantgoyal Jul 31, 2015
3632f2c
Commit for #97809356 : aidr-task-manager logging
Jul 31, 2015
3282770
Added logs in AIDR-Common.
kushalkantgoyal Jul 31, 2015
2b2080f
Commit for #97809356 : aidr-tagger logging
Jul 31, 2015
c643e06
Added logs in Aidr-Manager.
kushalkantgoyal Aug 1, 2015
0c2c31e
Commit for #97809356 : aidr-db-manager logging
Aug 1, 2015
9cd58ee
Added logs to AIDR-TrainerAPI.
kushalkantgoyal Aug 1, 2015
be7a249
Commit for #97809356 : aidr-tagger-api logging
Aug 3, 2015
cd4493e
Minor code fix.
kushalkantgoyal Aug 3, 2015
d1687cd
Commit for fixing deployment issues in aidr-analytics and db-manager
Aug 3, 2015
2e2222f
Commit for logging : changing logger instance to static final in dtos
Aug 3, 2015
fd30985
Minor changes. Changed logger to static final
kushalkantgoyal Aug 3, 2015
a26e5d9
tagger: retrain model y/n moved back to sys.out
meghnas Aug 4, 2015
51dfc78
Added some logs in manager and fixed a bug in JsonDeserializer
kushalkantgoyal Aug 4, 2015
fa43755
Merge branch 'dev' of https://github.com/qcri-social/AIDR into dev
kushalkantgoyal Aug 4, 2015
f1f5eff
Fixed download link for filter/genCsv
kushalkantgoyal Aug 4, 2015
83e228c
Added a check on ui to report 0 human tagged tweets as info not error.
kushalkantgoyal Aug 4, 2015
1d720ec
Fixed csv download link null issue if there wasn't any tweet.
kushalkantgoyal Aug 4, 2015
3edfe48
Fixed csv download link null issue for tweetIds if there wasn't any t…
kushalkantgoyal Aug 4, 2015
03eeccf
Merge pull request #40 from qcri-social/dev
kushalkantgoyal Aug 5, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions aidr-analytics/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@
<artifactId>aidr-common</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<!-- dependency>
<groupId>qa.qcri.aidr.output</groupId>
<artifactId>aidr-output</artifactId>
<version>1.0</version>
<type>jar</type>
</dependency>
</dependency -->

<!-- Web dependencies -->

Expand Down
2 changes: 1 addition & 1 deletion aidr-analytics/src/main/java/qa/qcri/aidr/analysis/api/ApplicationConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
*
* @author Imran
*/
@javax.ws.rs.ApplicationPath("rest")
@javax.ws.rs.ApplicationPath("/rest/analytics/")
public class ApplicationConfig extends Application {

@Override
Expand Down
22 changes: 1 addition & 21 deletions aidr-analytics/src/main/java/qa/qcri/aidr/analysis/api/GetConfidenceStatistics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@

import qa.qcri.aidr.analysis.facade.ConfidenceStatisticsResourceFacade;
import qa.qcri.aidr.common.code.DateFormatConfig;
import qa.qcri.aidr.output.getdata.ChannelBufferManager;


/**
Expand All @@ -31,7 +30,7 @@
*/

@Path("/confData/")
public class GetConfidenceStatistics extends GetStatistics implements ServletContextListener {
public class GetConfidenceStatistics implements ServletContextListener {

// Debugging
private static Logger logger = Logger.getLogger(GetConfidenceStatistics.class);
Expand Down Expand Up @@ -64,25 +63,6 @@ public Response ping() {
return Response.ok(obj).build();
}


@GET
@Path("/manage/restart/{passcode}")
@Produces("application/json")
public Response restartAnalyticsService(@PathParam("passcode") String passcode) {
logger.info("Request received");
if (passcode.equals("sysadmin2013")) {
if (masterCBManager != null) {
masterCBManager.close();
}
masterCBManager = new ChannelBufferManager();
masterCBManager.initiateChannelBufferManager(CHANNEL_REG_EX);
logger.info("aidr-output analytics service restarted...");
final String statusStr = "{\"aidr-output analytics ConfidenceStats service\":\"RESTARTED\"}";
return Response.ok(statusStr).build();
}
return Response.ok(new String("{\"password\":\"invalid\"}")).build();
}

@Override
public void contextDestroyed(ServletContextEvent arg0) {
// TODO Auto-generated method stub
Expand Down
3 changes: 0 additions & 3 deletions aidr-analytics/src/main/java/qa/qcri/aidr/analysis/api/GetStatistics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package qa.qcri.aidr.analysis.api;

import qa.qcri.aidr.output.getdata.ChannelBufferManager;


public abstract class GetStatistics {
Expand All @@ -9,6 +8,4 @@ public abstract class GetStatistics {
public static final String CHANNEL_REG_EX = "aidr_predict.*";
public static final String CHANNEL_PREFIX_STRING = "aidr_predict.";

public volatile static ChannelBufferManager masterCBManager = null; // managing buffers for each publishing channel

}
70 changes: 57 additions & 13 deletions aidr-analytics/src/main/java/qa/qcri/aidr/analysis/api/GetTagDataStatistics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
package qa.qcri.aidr.analysis.api;


import javax.ejb.EJB;
import javax.json.Json;
import javax.json.JsonObject;
import javax.servlet.ServletContextEvent;
Expand All @@ -19,19 +20,62 @@

import org.apache.log4j.Logger;


import qa.qcri.aidr.analysis.facade.TagDataStatisticsResourceFacade;
import qa.qcri.aidr.analysis.service.GetTagDataStatisticsService;

/**
* This is the REST API interface for accessing the aidr_analytics DB's tag_data entity.
*/

@Path("/tagData/")
public class GetTagDataStatistics extends GetStatistics implements ServletContextListener {
/**
* @author koushik
*
*/
@Path("/tagdata/")
public class GetTagDataStatistics implements ServletContextListener {

// Debugging
private static Logger logger = Logger.getLogger(GetTagDataStatistics.class);

@EJB
private GetTagDataStatisticsService tagDataService;


/**
* @param crisisCode
* @param granularity
* @param startTime
* @return Tag counts for individual attributes of a crisis, only for the specified granularity and since 'startTime'
*/
@GET
@Path("/getlabelsum/{crisisCode}/{granularity}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountSumForAllAttributesFromTime(@PathParam("crisisCode") String crisisCode,
@PathParam("granularity") Long granularity,
@DefaultValue("0") @QueryParam("startTime") Long startTime) {

JSONObject json = tagDataService.getTagCountSumForAllAttributesFromTime(crisisCode, granularity, startTime);
return Response.ok(json.toJSONString()).build();
}



/**
* @param crisisCode
* @param startTime
* @return Tag counts over all attributes for a crisis, sorted by granularity levels
*/
@GET
@Path("/getlabelsum/{crisisCode}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountSumForAllGranularitiesFromTime(@PathParam("crisisCode") String crisisCode,
@DefaultValue("0") @QueryParam("startTime") Long startTime) {

JSONObject json = tagDataService.getTagCountSumByGranularity(crisisCode, startTime);
return Response.ok(json.toJSONString()).build();
}


/**
*
* @param crisisCode
Expand All @@ -41,14 +85,14 @@ public class GetTagDataStatistics extends GetStatistics implements ServletContex
* @return Count sum per label from startTime to current time at the given granularity, for crisisCode and attributeCode
*/
@GET
@Path("/getLabelSum/{crisisCode}/{classifierCode}/{granularity}")
@Path("/getlabelsum/{crisisCode}/{attributeCode}/{granularity}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountSumFromTime(@PathParam("crisisCode") String crisisCode,
@PathParam("attributeCode") String attributeCode,
@PathParam("granularity") Long granularity,
@DefaultValue("0") @QueryParam("startTime") Long startTime) {

JSONObject json = new GetTagDataStatisticsService().getTagCountSumFromTime(crisisCode, attributeCode, granularity, startTime);
JSONObject json = tagDataService.getTagCountSumFromTime(crisisCode, attributeCode, granularity, startTime);
return Response.ok(json.toJSONString()).build();
}

Expand All @@ -61,14 +105,14 @@ public Response getTagCountSumFromTime(@PathParam("crisisCode") String crisisCod
* @return Count per label in the specified time window at the given granularity, for crisisCode and attributeCode
*/
@GET
@Path("/getLabelCount/{crisisCode}/{attributeCode}/{granularity}")
@Path("/getlabelcount/{crisisCode}/{attributeCode}/{granularity}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountInTimeWindow(@PathParam("crisisCode") String crisisCode,
@PathParam("attributeCode") String attributeCode,
@PathParam("granularity") Long granularity,
@DefaultValue("0") @QueryParam("startTime") Long startTime) {

JSONObject json = new GetTagDataStatisticsService().getTagCountInTimeWindow(crisisCode, attributeCode, granularity, startTime);
JSONObject json = tagDataService.getTagCountInTimeWindow(crisisCode, attributeCode, granularity, startTime);
return Response.ok(json.toJSONString()).build();
}

Expand All @@ -82,7 +126,7 @@ public Response getTagCountInTimeWindow(@PathParam("crisisCode") String crisisCo
* @return Time series data for each label in the interval [startTime, endTime] at the given granularity, for crisisCode and attributeCode
*/
@GET
@Path("/getLabelTimeSeries/{crisisCode}/{attributeCode}/{granularity}")
@Path("/getlabeltimeseries/{crisisCode}/{attributeCode}/{granularity}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountTimeSeries(@PathParam("crisisCode") String crisisCode,
@PathParam("attributeCode") String attributeCode,
Expand All @@ -93,7 +137,7 @@ public Response getTagCountTimeSeries(@PathParam("crisisCode") String crisisCode
if (null == endTime || endTime < startTime) {
endTime = System.currentTimeMillis();
}
JSONObject json = new GetTagDataStatisticsService().getTagCountTimeSeries(crisisCode, attributeCode, granularity, startTime, endTime);
JSONObject json = tagDataService.getTagCountTimeSeries(crisisCode, attributeCode, granularity, startTime, endTime);
return Response.ok(json.toJSONString()).build();
}

Expand All @@ -107,15 +151,15 @@ public Response getTagCountTimeSeries(@PathParam("crisisCode") String crisisCode
* @return Count sum for each label in the interval [startTime, endTime] at the given granularity, for crisisCode and attributeCode
*/
@GET
@Path("/getIntervalLabelSum/{crisisCode}/{attributeCode}/{granularity}")
@Path("/getintervallabelsum/{crisisCode}/{attributeCode}/{granularity}")
@Produces(MediaType.APPLICATION_JSON)
public Response getTagCountSumInInterval(@PathParam("crisisCode") String crisisCode,
@PathParam("attributeCode") String attributeCode,
@PathParam("granularity") Long granularity,
@DefaultValue("0") @QueryParam("startTime") Long startTime,
@QueryParam("endTime") Long endTime) {

JSONObject json = new GetTagDataStatisticsService().getTagCountSumInInterval(crisisCode, attributeCode, granularity, startTime, endTime);
JSONObject json = tagDataService.getTagCountSumInInterval(crisisCode, attributeCode, granularity, startTime, endTime);
return Response.ok(json.toJSONString()).build();
}

Expand All @@ -130,14 +174,14 @@ public Response getTagCountSumInInterval(@PathParam("crisisCode") String crisisC
* @return The count for a label in the specified time window at the given granularity, for crisisCode and attributeCode
*/
@GET
@Path("/getOneLabelData/{crisisCode}/{attributeCode}/{labelCode}/{granularity}/{startTime}")
@Path("/getonelabeldata/{crisisCode}/{attributeCode}/{labelCode}/{granularity}/{startTime}")
@Produces(MediaType.APPLICATION_JSON)
public Response getSingleItem(@PathParam("crisisCode") String crisisCode,
@PathParam("attributeCode") String attributeCode,
@PathParam("labelCode") String labelCode,
@PathParam("granularity") Long granularity,
@PathParam("startTime") Long startTime) {
JSONObject json = new GetTagDataStatisticsService().getSingleItem(crisisCode, attributeCode, labelCode, granularity, startTime);
JSONObject json = tagDataService.getSingleItem(crisisCode, attributeCode, labelCode, granularity, startTime);
return Response.ok(json.toJSONString()).build();
}

Expand Down
Loading