Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes.
It is possible to get multiple ROC curves on a single plot by using the KnowledgeFlow. Set up a flow as follows:
ArffLoader -- dataSet -- > ClassAssigner -- dataSet -- > ClassValuePicker -- dataSet -- > CrossValidationFoldMaker -- trainingSet/testSet (i.e. BOTH connections) -- > Classifier of your choice -- batchClassifier -- > ClassifierPerformanceEvaluator -- > ModelPerformanceChart
Next configure the ArffLoader with a data set, and then configure the ClassValuePicker by selecting which class value you want to treat as the positive class. Then you can start the flow running by selecting the "start loading" action from the ArffLoader. The model performance chart will show the ROC curve.
Subsequent ROC curves for different algorithms can be displayed on the same plot by either deleting your first classifier from the existing flow and inserting a new one and then running the flow again, or, setting up a new flow identical to the first one (but with a different classifier) and connecting this one's ClassifierPerformanceEvaluator to the first flow's ModelPerformanceChart.
For subsequent curves to appear on the same plot the dataset, class attribute, positive class value and evaluation method (eg 10 fold Xval) must be the same as for the first curve.
Source: Weka FAQ